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EXECUTIVE SUMMARY 


PROJECT MISSION 

New academic demands and challenges in schools require increased capacity to support 
instructional improvement. Capacity comes in many forms: technical, organizational, and 
political. In addition, there are different sources of capacity, such as the state government and 
networks. This report examines: 


What sources of capacity to support instructional improvement operate in California? 
How are they distributed? 

How does California’s capacity compare with other states? 

How might California’s capacity be strengthened? 


We specifically examined these questions of capacity relative to puzzles of creating coherence 
and instructional alignment for California’s ambitious grade-level standards. 


CURRENT CONDITIONS: TECHNICAL, ORGANIZATIONAL, AND POLITICAL CAPACITY 
Our research identified the following prominent features in the current landscape of technical, 
organizational, and political capacity around instructional support. 


Calls to “stay the course”: “Stay the course” with respect to the main policy elements— 
subject-matter standards, assessments, LCFF/LCAP—and more time to put them into 
practice emerged as consistent themes across interviews with superintendents and with 
state/regional leaders. 


Calls for help managing “the noise”: Along with general support for subject-matter 
standards, come calls for help curating instructional materials, professional 
development, and the “noise in California” from non-profit and for-profit education 
service providers. 


Many districts look to the state for help: Our interviews revealed high percentages of 
superintendents looking to the state for information on instructional materials and on 
LCFF and LCAP implementation, especially among superintendents from districts with 

high concentrations of ELL students. 


Current conditions in the CDE constrain the agency’s ability to support frontline practice. 


Limited CDE in-house subject-matter expertise: Reductions in CDE staff have occurred 
disproportionately in portions of the agency devoted to instructional support. 


Greater staff reductions in California than in other states: State-level staff reductions 
over time have been significantly higher in California than in other states. 


Lower average salaries for state-level positions: One challenge to attracting and 
retaining subject-matter experts arises from lower average salaries in the CDE than in 
high enrollment county and district offices. 
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Current conditions in the terrain of instructional support challenge coherence. 
e Expanding sector of non-profit providers: Like other states, California has experienced 
an expanded terrain of non-profit service providers. 


e Inequities in distribution of providers: The distribution of California’s non-profit 
education service providers exhibits geographic disparities: a lack of proximity to 
providers in some areas, and an under-provision of services relative to student 
enrollment and poverty rates in other areas. 


e Network challenges connecting and curating: California’s vibrant space of networks 
offers one way of reaching beyond geographic limits. Interviews revealed the promise of 
networks for distributing information and resources but also challenges of durability, 
isolation, and hyper-connectivity. 


OPPORTUNITIES TO LEARN 
Evidence from current conditions and from models from other states and other agencies 
suggest some opportunities for California to learn as it strives to build coherence across the 
elements of instructional infrastructure. 
e To build technical capacity: 
o Other agencies provide models for how to adjust state salaries to align with 
California’s public-sector labor market; 


o Exchanges that bring county or district-level staff to the state level have the 
potential to cultivate flexible, current subject-matter expertise at the state level. 


e To build organizational capacity: 
o Existing subject-specific network strategies like the California Subject Matter 
Project and the Communities of Practice provide foundations through which to 
connect subject-matter content with improvement processes; 


o Other states provide examples for how to build learning through in-house 
strategy and research. 


e To build political capacity: 
o Interviews make clear that implementers at all levels of the system want time 
and space and support to stay the course for major policy components— 
standards, assessments, and Local Control Funding Formula (LCFF). 
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State Structures for Instructional Support in California 


1.0 INTRODUCTION 

“This changes almost everything,” State Board of Education President Mike Kirst said of the 
Common Core State Standards (CCSS). “The CCSS will impact almost all key state education 
policies in fundamental ways” (Kirst 2013, 1). California’s move towards encouraging ambitious 
instruction aligned with the Common Core State Standards occurred alongside several other 
significant disruptions to established policy and practice. Putting ambitious grade-level 
standards into instructional practice coincided in California with a statutory shift away from 
state-level categorical funding and toward greater local discretion over funding priorities and 
use. This move toward decentralization also coincided with the lingering effects of the Great 
Recession of 2008 and with an expanding sector of non-governmental organizations active in 
California’s educational terrain. Together, these ambitious standards, shifting governance 
responsibilities, constrained resources, and expanding terrain of support providers yielded 
heady and hair-raising conditions for instructional practice. Along with enthusiasm for 
California’s policy changes over the past decade have come calls to “stay the course” to enable 
the current changes to make their way into practice: 


“| hope we stay the course in California for a while. | feel like we have made 
strides in this very short time frame towards improvements. Is it a perfect 
system? No, but | think there's enough places where improvements have begun 
to happen, staying with them helps. We need the same assessments. We need 
the standards, we need the funding system and Dashboard to take hold in the 
best way so that we continue to improve” (State/Regional Leader Interview 006). 


The state/regional leader’s perception that “I feel like we have made strides in this very short 
time frame towards improvements” parallels teachers’ perspectives. Evidence from a survey of 
teachers’ perspectives that we discuss in detail elsewhere in the GDTF paper series (Moffitt et 
al. 2018) reveals that California teachers’ modal response is “improved a little” to a series of 
questions about key components of instruction: alignment between standards and materials, 
alignment between district professional learning opportunities and teacher needs, the quality 
of instructional materials, the quality of professional learning opportunities, and the quality of 
school-level learning communities. Results suggest teachers from districts with high 
concentrations of English Learners are more likely to report perceived improvement over the 
past three years than teachers from districts with low concentrations of English Learners for key 
aspects of instructional resources. However, teachers from schools with higher concentrations 
of poverty or higher concentrations of students who are English Learners were significantly less 
likely to perceive that their instructional materials are well suited to the needs of their students 
than teachers in low poverty, low-ELL schools. 


These teachers’ perspectives lend support for calls to “stay the course in California” so “we 


continue to improve.” Instructional support “so that we continue to improve” depends on 
technical, organizational, and political capacity. It depends on technical capacity to generate 
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and share the know-how to put California’s ambitious grade-level standards into instructional 
practice deeply and equitably. It depends on organizational capacity to coordinate instructional 
support across California’s vast and varied terrain. It depends on political capacity to allow the 
process of putting standards into practice to “stay the course.” 


In this era of decentralization and constrained resources, capacity “so that we continue to 
improve” arises from both governmental and non-governmental sources.? Chief among the 
governmental entities responsible for supporting instructional improvement are county offices 
of education (Plank 2018). The governor’s 2018/2019 budget invests in county offices to 
provide instructional support (Governor’s Budget Summary 2018-2019 K-12 Education, 26). 
Non-governmental sources of instructional support, however, have become increasingly 
prominent in California and nationally over the past twenty-five years. While state capacity for 
instructional support encompasses much more than the CDE’s capacity, the CDE (under the 
direction of the Superintendent of Public Instruction) remains a vital component of instructional 
support with its responsibility to gather and distribute data, to support the development of 
curriculum frameworks, and oversee county offices of education (LAO 2018, 5).* 


Governmental and non-governmental organizations can operate as zero-sum competitors: 
more authority and resources for one can mean less for another. However, levels of 
government and organizational actors can operate in positive-sum terms: expanding 
responsibility at one level can simultaneously expand responsibility at another (Cohen 1982). As 
California looks forward to the next administration, it has the opportunity to systematically 
construct its structures of instructional support as complementarities (Brass 2014): the central 
state can support decentralized decision-making and practice. The analysis below reveals 
conditions and examines other state models for cultivating and sustaining structures of 
instructional support “so that we continue to improve.” 


Purpose of this report 
New academic demands and challenges in schools require increased capacity to support 


instructional improvement. Capacity comes in many forms: technical, organizational, and 
political. In addition, there are different sources of capacity, such as the state government and 
networks. This report examines: 

e What sources of capacity to support instructional improvement operate in California? 

e How are they distributed? 

e How does California’s capacity compare with other states? 

e How might capacity be strengthened? 


1 Throughout this report, we deliberately distinguish between governmental and non-governmental sources of 
support instead of relying on the term “intermediary.” As we argue elsewhere (Cohen and Moffitt 2009), 
positional terms are rubbery categories: an intermediary in one task may be a policymaker for another. We focus 
on governmental and non-governmental because of their implications for democratic accountability. 

* List of responsibilities from Legislative Analyst’s Office (LAO) report: “Compile and disseminate data on districts, 
schools, staff, and students”; “Oversee development of curriculum frameworks, standardized student assessments, 
instructional materials, and school facilities standards”; “Oversee county offices of education” (LAO 2018, 5). 


2 | State Structures for Instructional Support in California 


As it addresses these questions, this report strives: 

e To build on but not repeat prior analysis of California’s education governance. We 
encourage readers to consult comprehensive reviews (Timar 2004; Brewer 2007) and 
prior studies of California’s education governance for historical details and a long 
tradition of reform efforts in California. These prior studies and reviews include: the 
California Constitution Revision Commission (1996); the Joint Committee to Develop a 
Master Plan for Education (2002); the California Performance Review (2004); the 
Governor’s Committee on Educational Excellence (2007); Little Hoover Commission 
(2008). 


e To focus on state structures for instructional support, but not consider all aspects of 
educational governance capacity. To this end, the report will focus on standards, 
curriculum, frameworks, materials, and professional development. These comprise 
some of the main components of infrastructure for instruction (Cohen and Moffitt 
2009). 


e To focus primarily on state-level structures for support, with particular attention on the 
California Department of Education. Other reports in this series will focus on different 
governing entities, including county offices of education and school boards, the 
Dashboard, and local entities involved with Local Control Funding Formula/Local Control 
and Accountability Plan (LCFF/LCAP) implementation. Our conceptual approach, 
nevertheless, understands state capacity as encompassing more than any singular 
governing entity. 


This report considers opportunities to learn from other agencies, from other states, and from 
other levels of government. In doing so, it takes several conditions as given: 
e It takes the constitutional structure of California’s education governance as a given: it 
does not attempt to propose reforms that would require constitutional changes. 


e It takes financial constraints as a given. The lingering effects of the 2008 Great Recession 
remain palpable, fiscal burdens from pensions loom, and funding constraints appear as 
a top concern to California district superintendents. 


e Ittakes the recent move to decentralization as a given. Though some calls for a return to 
categoricals have emerged, we assume that the basic architecture of LCFF/LCAP will 
remain in place for the near-term. 


3 It is important to note that the State of California in its most recent fiscal year 2018 budget has adopted language 
with intent to pursue legislation that would improve the Local Control and Accountability Plan (LCAP). For specific 
references see page 25 here: http://www.ebudget.ca.gov/2018-19/pdf/Enacted/BudgetSummary/K- 
12Education.pdf and see page 46 here: http://www.dof.ca.gov/Budget/Trailer_ Bill Language/documents/2018- 


19EducationOmnibusTrailerBill 001.pdf. 
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2.0 CONCEPTS, TERMS, AND OVERVIEW OF KEY COMPONENTS OF CAPACITY 
How might one build a coherent system, in the context of the ideas contained in the Common 
Core State Standards Initiative that change “almost everything”? What pieces would need to be 
coordinated and what technical, organizational, and political puzzles must be solved to do so? 


2.1 What Do We Mean by Structures for Instructional Support and Coherence? 

Any satisfactory framework for building a coherent system must deal with at least three 
matters. One is to specify the nature of instruction, since instruction is the object of efforts to 
encourage coherence. The second is to define the organization of the education sector, since it 
is the actions of agencies and individuals in this sector that would influence coherence. The 
third is the design of the Common Core State Standards Initiative, since this is the agent that 
aims to create coherence in instruction in the education sector. 


With respect to the nature of instruction, our work presumes that it is a multi-component social 
technology that gains in effectiveness when its various components (academic standards, 
assessments, instructional materials, and teacher education/professional development) are 
aligned among themselves so as to produce coherent—i.e., internally consistent—guidance for 
the academic work that students and teachers do (Cohen and Spillane 1992; Correnti and 
Rowan 2007; Cohen et al. 2014). 


As we consider the parts of the system that need to be coordinated and the technical, 
organizational, and political puzzles that must be solved, we must turn to the organization of 
the education sector in the U.S., since that is where agencies and individuals that might affect 
instruction reside. The following key structural features of this sector are not unique to 
California but instead reflect the sector more broadly in the United States. These features 
include: 

e Multiple levels (following the federal structure of government); 


e Strong traditions of, and statutory support for, local control; 

e Aweak public sector and an influential non-governmental sector (owing to built-in and 
inherited constraints on government and to the historically large and still-growing role 
that private sector agencies play in public education); 

e The presence of multi-level vertical “silos” in and out of government that track federal 
and state policies and contribute to the weakness of state and local general 


governments; 


e The long-standing absence of any capacity for monitoring the quality of instruction and 
a related lack of capacity for improving instruction; 


e An occupation of teaching that never developed either the deep technical knowledge 
and skill or control of the workplace that are associated with strong professions. 
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These structural features of the education sector interact with deeply entrenched structural 
inequalities that manifest in the U.S. along racial, ethnic, and socio-economic lines. 


One functional consequence of these structural conditions has been the pervasive absence of 
the capacity, in agencies of all sorts across the United States, to offer strong and consistent 
guidance for instruction. Though pervasive, this capacity problem is distributed and 
experienced unequally, again owing to structural inequalities in the United States. The relevant 
agencies include those that offer teacher education and professional development, that 
develop curriculum, and that design assessments, as well as many others that offer more 
specialized or episodic guidance. Such guidance has typically been fragmented, often divergent, 
and weak. Though the volume of guidance grew through the 20" century, that growth was not 
accompanied by increased consistency (Cohen and Spillane 1992; Cohen et al. 2014). 


A second consequence has been little capacity for coordinating the action of organizations in 
the education sector. Government has been internally divided among three federal levels and 
three branches; vertical silos have been relatively strong; and though there are many non- 
government agencies, they typically have been too specialized to exert much coordination 
(Cohen and Moffitt 2009). A third consequence has been variable but generally weak capacity 
for both instruction and the improvement of instruction. The weakness of teaching and teacher 
education, the fragmented guidance for instruction, and the lack of capacity for monitoring the 
quality of instruction all contributed to the absence of much capacity for improving instruction. 
Until very recently neither monitoring nor improving the quality of instruction were functions 
to which state and local governments gave much attention. 


This summary presents two key elements of the theory that informs our work. It also leads us 
toward preliminary answers to questions about what pieces would need to be coordinated and 
what technical, organizational, and political puzzles must be solved to do so. In very reduced 
form, the parts of the system that would need to be coordinated are those that have a strong 
influence on the key components of guidance for instruction, namely academic standards, 
assessments, instructional materials, and teacher education/professional development. Given 
our summary account of the organization of the education sector just above, there are 
imposing technical, organizational, and political puzzles to solve to coordinate parts of the 
system. 


These puzzles can usefully be summarized in a few questions: 

e How to coordinate the actions and products of many independent private sector 
assessment designers and publishers so that they assess the same knowledge and 
skills? This is a puzzle because at present the two CCSSI assessment consortia are a 
small minority of all the commercially available and widely used assessments, there is 
no reason to think that there is much “alignment” among them, and at present there 
exist no organizational means within the sector to coordinate these agencies. 

e How to coordinate the actions and products of many independent private sector 
curriculum designers, publishers, and materials providers so that they deal with the 
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same knowledge and skills? This is a particular puzzle because at present there exist no 
organizational means within the sector to coordinate the many varied and often quite 
specialized curricula and materials. 


How to coordinate the actions and products of many independent private and public 
sector teacher education and professional development agencies so that they instruct 
teachers in the same knowledge and skills? This is a puzzle because at present there 
exist only weak means to coordinate the work of these schools, departments, and 
programs. 


How to develop the means to monitor instructional quality, and the means to use the 
knowledge that results to improve instruction? This is a puzzle because at present 
there exist very few agencies that do such monitoring, because most monitoring of 
instruction is instead monitoring of outcomes, and because very few agencies have 
undertaken to use monitoring of instruction in instructional improvement. 


How to coordinate these four functions, and the agencies that perform them, so that 
they attend to the same knowledge and skills? This is a grand puzzle. One reason is that 
researchers have only recently begun to explore how to define and measure 
“alignment” —we would write consistency—either among curricula or among 
assessments, let alone among teacher education programs. Prior to standards-based 
reform there were few incentives to even attend to such issues. Another reason is, even 
if the definition and measurement problems were solved, there never have been 
agencies that even contemplated such coordination in the education sector. 


How to coordinate these four functions, and the agencies that perform them, when 
the school systems that offer instruction are so unequally resourced, and when those 
systems reside in varied racial, ethnic, linguistic, and socio-economic contexts? 


There are several ways to solve these puzzles. One would be to build school systems with 


coherent instruction and instructional guidance, on the assumption that the school system is 


the chief unit of action, and that coherence can best and most durably be built at that level. 


Some systems, including some Montessori, International Baccalaureate, some charter systems, 
some Comprehensive School Reform Designs, and some LEAs have made significant progress on 


several of the puzzles by building such internally coherent systems of instruction and 
instructional guidance (Cohen et al. 2014). 


Another solution would be to restructure the environment of the education sector so that it 
offered pressure and support for coherence, on the assumption that unless that environment 


were changed the systems and other organizations within the sector could not sustain the 


arduous work of building coherence. Pre-CCSSI versions of standards-based reform took this 
route and made some progress on some of the puzzles by creating an exoskeleton of standards, 
assessments, and accountability that were intended to encourage or “drive” school systems to 


affect coordination. 
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Still another way to solve the puzzle would be to focus on the historically very influential non- 
governmental sector, on the assumption that since it operates outside the complicated realms 
of democratic governance, non-government agencies might be able to more effectively 
leverage coherence in the sector. And several non-government agencies—foundations, 
Achieve, and others—have tried to offer incentives and support for local and state efforts to 
solve some of the puzzles, and they have had some influence. 


The CCSS Initiative has taken a somewhat different approach. It presented a rudimentary 
theory of action in an early publication (Benchmarking for Success, 2008). That document 
envisions states combining to use their political and market influence to create coherence in 
curricula, to create coherence between curricula and assessments, and to create coherence 
among curricula, assessments, teacher education, and national standards. The instruments that 
the CCSS Initiative controls are standards and assessments. It will remain for adopting states to 
use their influence to encourage private sector and non-profit agencies to use the national 
ideas to serve the public purposes of improved instructional quality, coherence, and reduced 
inequality. Benchmarking for Success and many subsequent CCSS Initiative publications say 
nothing about the weak capacity for instruction and instructional improvement, especially in 
the context of structural inequalities that manifest along racial, ethnic, linguistic, and socio- 
economic lines. 


Given the great novelty and dynamic nature of standards-based reforms and the CCSS Initiative 
incarnation of those reforms, a single-minded theory of coherence is not tenable. Instead, we 
focus here on how the technical, organizational, and political puzzles have been unfolding in 
California and what they suggest for the path forward. 


2.2 What Do We Mean by State Capacity? 

This report focuses, conceptually, on “infrastructural capacity.”* Generally, this means the 
collective capability of governmental and non-governmental actors to put ideas into practice 
consistent with state goals (Mann 1984): specifically, the uptake of Common Core ideas in 
instructional practice. This approach to capacity is fundamentally relative. As Sikkink argues 
“state capacity is a relational concept: the state is strong or capable in relation to the tasks it 
sets for itself” (1991, 11). Capacity is not absolute but depends on the task that state aims to 
accomplish. While number of staff members and amounts of funding can serve as important 
proxies, their contributions to infrastructural capacity depend the scope and ambition of what 
the staff and their resources are expected to accomplish. 


This approach is also fundamentally collective: it considers the full sum of organizational 
contributions to performing tasks, including local capacity, rather than only the distinct capacity 


4 Infrastructural power differs from coercive power (such as military force) or extractive power (such as taxation). 
On this distinction, see Weber, Economy and Society 1969 originally 1922, Gunther Roth and Claus Wittich trans. 
New York: Bedminster Press, vol 2, chapter 9, vol 3, chaps 10-13, cited in Skocpol 1985, 33. 
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of the central state (Mann 1984; Ziblatt 2008; Soifer and von Hau 2008; Steffes 2012).° As we 
have argued elsewhere, the state, its social policies and practice embody mutual dependence.°® 
Policy defines problems and specifies goals for social change — such as less crime, more 
employment, or higher educational achievement. The targets of social policy — the individuals 
and organizations that policy defined as the problem — depend on the rules, incentives, 
knowledge, financial resources that policy offers to change practice. At the same time, 
policymakers ultimately depend on those target individuals and organizations to make use of 
the rules and resources that policy offers, if they are to move practice closer to the goals that 
policy specifies. Policy depends on students to learn, on teachers to teach, on doctors to treat, 
on polluters to reduce emissions. 


We focus on three aspects of infrastructural capacity: 
* Technical capacity, which consists of the knowledge and know-how to put policy into 
practice.’ 
* Organizational capacity, which consists of position-based authority and responsibilities, 
and relationships between constituent and collaborative units. 
* Political capacity, which consists of durable support from groups, elected officials, and 
constituents. 


2.3 In the Context of California’s Instructional Support, Who Does this Report Include in 
Collective Capacity? 

State capacity to put policy into practice ultimately depends on front-line actors including 
teachers, families, and students (Sullivan and Strach 2011). For the purposes of this report, 
however, we are focusing on state-level and state-wide governmental and non-governmental 
organizations, with a particular attention to the CDE. 


The elected State Superintendent of Public Instruction leads the CDE, which is broadly 
responsible for overseeing “the state's diverse public school system.”® This entails 
administrative responsibility for approximately 6.2 million students, 10,000 schools, 295,000 
teachers, and 1,000 districts. Roughly 1 in 8 American public school children attend California 
public schools. To put the enormity of California’s responsibility in comparative perspective, 
California educates more public school children than the entire country of Canada. 
In terms of instructional support, the CDE has more specific responsibilities to: 

e “compile and disseminate data on districts, schools, staff, and students” 


5 As Ziblatt (2008) reminds us, “in federal political systems, highly capable local governments—with resources, 
expertise and professionalism— might represent a “decentralized” or “bottom-up” path for achieving higher 
overall levels of state infrastructural power in a political system.” This approach recognizes that subnational 
weakness can impair the strength of the state (Weir 2005). 

5 For an elaboration on this point, see Cohen and Moffitt (2009). 

7 This is sometimes referred to as “intellectual talents” (Skowronek 1982, 19; Sikkink 1991). This also includes 
values (Meier and O’Toole 2006). 

8 California Department of Education Website: Homepage, https://www.cde.ca.gov/, accessed on December 7, 
2017. 
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e “oversee development of curriculum frameworks, standardized student assessments, 
instructional materials, and school facilities standards” 
e “oversee county offices of education” (LAO 2018, 5). 


Three CDE branches with clear ties to instructional support include: 

e Performance, Planning, and Technology Branch: “Oversees programs promoting 
innovation and improved student achievement. Programs include statewide student 
assessment, school and district interventions, state and federal accountability, collection 
and reporting of educational data, and charter schools.” 


e Teaching and Learning Support Branch: “Promotes the policy and budget priorities of 
the State Superintendent of Public Instruction before the Legislature, Congress, and the 
state and federal Administrations, and provides guidance and support for the 
development and oversight of high quality charter schools.” 


e Systems Support Branch: “Oversees the CDE budget, accounting, information systems, 
and personnel services; apportionment of state and federal resources to local 
educational agencies (LEAs); and assistance to LEAs for fiscal can business aspects of 
public schools.”? 


Other governmental entities play pivotal roles in the provision of instructional support, 
including the California Board of Education (CBE), the Commission on Teacher Credentialing 
(CTC), the California Collaborative for Educational Excellence (CCEE), the county offices of 
education, and the counties’ association the California County Superintendents Educational 
Services Association (CCSESA). In the sections that follow, we consider the CDE’s relationship 
with other state-wide organizational entities. Both governmental and non-governmental 
entities combine to form California’s vibrant terrain of networks, which we also consider in the 
sections that follow. 


2.4 What Do We Mean by Networks? 

Networks, collaboratives, and communities of practice are a well-established form of building 
professional expertise and sharing ideas in California’s educational terrain. We focus on and 
define networks as a “form of collaborative activity for facilitating and operating multi- 
organizational arrangements to solve problems that cannot be solved or solved easily by using 
single organizations” (McGuire 2011, 437-438). We focus primarily on two sorts: information 
diffusion networks and service implementation networks. 


In educational contexts like California’s, networks can yield important assets. They can: 
¢ Disseminate information quickly 
* Promote innovation 
¢ Offer flexibility 


° California Department of Education Website: Organization, https://www.cde.ca.gov/re/di/or/, accessed on 
December 27, 2018. 
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¢ Approach non-routine, non-standard problems 
¢ Approach problems that are bigger than any single organization (Goldsmith and Eggers 
2004). 


Networks, however, are not unalloyed goods. Networks can also: 
* Cloud responsibility and accountability 
* Beunstable and difficult for their constituents to navigate 
¢ Offer limited or skewed coverage and access to network resources 
* Come with coordination costs 
* Come with monitoring costs 
¢ Yield hyper-connectivity and information overload (Goldsmith and Eggers 2004). 


Though differences emerge about what constitutes a network, collaborative, or community of 
practice, there has nonetheless been a proliferation of efforts that identify themselves in this 
way. Some network approaches generate significant attention and investment. The Carnegie 
Foundation for the Advancement of Teaching has been advocating for, leading, and supporting 
“Networked Improvement Communities” for the better part of the last decade (Bryk, Gomez, 
and Grunow 2011)?°; the Bill and Melinda Gates Foundation announced in November of 2017 
that their new K-12 strategy will focus on “networks for school improvement” (Bill and Melinda 
Gates Foundation 2017).*? 


Networks that bear on instructional support in California vary in size, scope, membership, 
purpose, and duration. Various snapshots of California’s networks include a 2015 convening of 
“California Implementation Networks” funded by the Bill and Melinda Gates Foundation 
(Convening of California Implementation Networks 2015)? and a list of current Carnegie 
Foundation for the Advancement of Teaching supported networks.?? In 2014, Danielle Hagood 
created the California K-12 Collaborative Network Inventory, identifying 61 networks operating 
in the state at that time. The list organized the networks into County Networks, Higher 
Education and K-12 Partnerships, Inter-County Networks, Inter-District Networks, and Regional 
Networks. Despite identifying so many networks, Hagood was clear the list was illustrative 
rather than comprehensive, noting many of the challenges that exist in trying to map this 
terrain. 


improvement-communities-education/ 
11 Bill and Melinda Gates Foundation, 2017: Outline for Our K-12 Stiaheny: 2017: 


improvement/, accessed May 21, 2018. 


Bill and Melinda Gates Foundation, 2015: Convening of California Implementation Networks, 
http://sccommits.org/sites/all/files/public/Convening%200f%20California%20Implementation%20Networks%20- 
%20eBinder.pdf, accessed May 21, 2018. 

13 Carnegie Foundation for the Advancement of Teaching Website: Current Work: Carnegie-Supported Networks, 


https://carnegie.staging.rygn.io/in-action/current-work/, accessed on February 17, 2018. 
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Some county offices and districts operate networks that do not have webpages or receive 
external funding. In interviews conducted for this report, two separate county offices of 
education suggested that they ran upwards of twenty networks within their counties. We 
discuss networks further in section 4.0 (Organizational Capacity) in this report. 


2.5 Methodological Approach 
We offer a mixed method approach to address our core questions: 
e What sources of capacity to support instructional improvement operate in California? 
e How are they distributed? 
e How does California’s capacity compare with other states? 
e How might capacity be strengthened? 


This includes: 
e Interviews of 44 state-level and regional-level actors 
e Interviews of 91 district superintendents, selected from a stratified random sample 
e Comparative analysis of state staffing patterns and salaries over time, both within 
California and across states?® 
e Comparative analysis of 376 non-profit education service providers in California and 
their distribution, over-time (2000 - 2015)?” 


15 


M4 Respondents, selected through snowball sampling, included a wide range of state and regional leaders from 
different levels of government and different sectors, including advocacy, research, and educational non-profit 
organizations. Interviews occurred between December 2016 and May 2018 and included 35 respondents based in 
California, and 9 based in other states or at the national level. Each semi-structured interview, lasted 45-120 
minutes, and was subsequently analyzed using a coding architecture informed by our research questions and 
refined over the process of analysis. 

15 This represents a 44.39% response rate of California district superintendents were drawn from a stratified 
random sample of 205 districts. All interviews occurred by telephone between June and October 2017, and lasted 
approximately 30 minutes. Interviews were analyzed based on a coding architecture derived from the interview 
protocol. Full detail appears in the Appendix. 

16 A full list of sources appears in the Appendix. 

1” The data on non-profit education service providers in California from 2000-2015 combines organizational data 
from the National Center for Charitable Statistics Core Trend Public Charities, county level demographic data from 
Census Bureau’s Small Area Income and Poverty Estimates (SAIPE), and aggregated data on student demographics 
from the California Department of Education. To create a sample of non-profit education service providers, we 
focused our analysis on organizations with the following classifications: research institutes and public policy 
analysis, special education, libraries, student services, educational services, and remedial reading and 
encouragement. The organization data was spatially merged with county demographic data using ArcGIS software. 
The dataset of education service organizations has 1,869 observations, or 376 unique organizations. 
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3.0 TECHNICAL CAPACITY 
“Common Core Implementation requires CDE staff to have more expertise 
than in past standards implementation efforts” 
(State/Regional Leader Interview 001). 


Putting California’s ambitious grade-level content standards into practice deeply and equitably, 
and aligning all parts of the system—curriculum, frameworks, instructional materials, 
professional development, and assessments—to support that instruction expects extraordinary 
changes in practice. We noted earlier that the United States has historically exhibited pervasive 
absence of the capacity to offer strong and consistent guidance for instruction (Cohen and 
Spillane 1992; Cohen et al. 2014). Much more variation resides in California than our broad- 
brush summary of the general U.S. terrain. 


California’s terrain includes deep commitment among leaders for the ideas embodied in the 
Common Core, a variety of organizational efforts aimed at supporting instructional practice 
aligned with California’s grade-level standards, and a pervasive plea to “stay the course” so 
putting the standards into practice can take root. California’s governmental and non- 
governmental structures to support instruction reflect deep commitments, considerable 
energy, and exemplary models of expertise. Yet, capacity is fundamentally relative to the task 
at hand. And, the enormity of the task and the scope of change in California loom large, in an 
era of ambitious standards. 


We focus here on technical capacity: the knowledge, know-how, and values to put policy into 
practice. Several themes emerge from the data and interviews: 
e Enormity of the task 
e Limited CDE resources relative to the enormity of the task 
e The importance of CDE as a source of instructional support, even in a terrain with other 
important providers 


3.1 Task Ambition Relative to Capacity: The Foundation on Which Current Efforts Build 

“So that we continue to improve” putting ambitious content standards into instructional 
practice deeply and equitably depends on coordinating the elements that have a strong 
influence on the key components of guidance for instruction, namely academic standards, 
assessments, instructional materials, and teacher education/professional development; the 
knowledge to coordinate those elements; and the political will to do so. “So that we continue to 
improve ...” our respondent continued: 


“... that's going to involve coaching ... and intentional technical assistance and 
real, in-depth examination of the instructional core around the teachers, the 
students, and the content. This examination needs to involve preparation 
programs as well so that they are built on similar assumptions” (State/Regional 
Leader Interview 006). 
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As SBE President, Mike Kirst, observed, a standards-aligned approach to instructional 
improvement “changes almost everything” (Kirst 2013, 1). As our respondent observed, 
standards come with significant and serious implications for the instructional core, so that 
content, technical assistance, and teacher preparation are both “in-depth” and “built on similar 
assumptions.” 


Yet, standards-based reform is not new to California; and parts of structures to support the 
instructional core began to take shape decades prior. California began laying the conceptual 
and policy foundation for greater coherence in California public education in the 1980s. One 
step in that process entailed defining what it meant to have quality curriculum and connect that 
with different leverage points in the system, including professional development.*® 


Professional Development and Subject Matter Support 
In 1988, the California state legislature passed Senate Bill 1882, the Professional Development 


Act. This act was an effort to foster “a comprehensive, consistent policy toward staff 
development to the end that each level of the elementary and secondary educational system 
has an effective staff development component tied directly to an organization improvement 
plan” (SB 1882 1988). SB 1882 provided funding for professional development, called for the 
creation of regional resource agencies and consortiums that would assist schools and districts in 
these efforts, and established a network of nine discipline-specific professional development 
projects called the California Subject Matter Project (CSMP). This bill was in response to the 
findings and recommendations of the California Staff Development Policy Study (Little et al. 
1987) that was initiated by the legislature and governor and called out the California Writing 
Project (CWP) and the California Mathematics Project (CMP) as examples of promising models 
in the state. 


The CWP and CMP were first funded by the California legislature in 1979 and 1982 respectively. 
These two subject matter projects were based on the Bay Area Writing Project (BAWP), which 
was created by James Gray at the University of California Berkeley in 1974. These projects 
brought teachers and university faculty members together in summer institutes in which 
teachers taught teachers and everyone engaged in the subject matter. With the passage of SB 
1882, the legislature sought to build upon and incorporate CWP and CMP into this larger 
network of nine subject matter projects: The California Arts Project, California History-Social 
Science Project, California International Studies Project, California Mathematics Project, 
California Physical Education-Health Project, California Reading and Literature Project, 
California Science Project, California World Language Project (formerly the California Foreign 
Language Project), and California Writing Project. 


The CSMP was reauthorized in 2011 by Senate Bill 612; and it is administered by the University 
of California, Office of the President in cooperation with the Concurrence Committee, which 


18 State/Regional Leader 011. 
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includes representatives from key state education agencies.’? Despite “changes in funding 
sources and significant declines in overall funding availability” (Bier and Gallagher 2012, 6), the 
CSMP still exists today and operates in roughly 90 regional sites across California. In addition, a 
ten-year evaluation of CSMP conducted by SRI International concluded that the CSMP “puts 
California in a unique position among the states of having a professional development 
infrastructure that can respond quickly and flexibly as needs change” (Bier and Gallagher 2012, 
7). The subject matter projects stand as an important, structural approach to providing 
professional learning opportunities through networks connected to higher education. Yet, 
California’s professional development terrain is much more vast and varied than the CSMPs 
alone. Along with this vast variation has come a history of inconsistent teacher exposure to 
quality professional development, and inconsistent opportunities to engage deeply with 
teaching rigorous content (Cohen 1990; Cohen and Hill 2001; Wilson 2003). 


Curriculum and Frameworks 

California’s Instructional Quality Commission (IQC)*° advises the State Board of Education on 
curriculum frameworks, on instructional materials, and on criteria for evaluating instructional 
materials. The l|QC also “recommends policies and activities to the SBE, California Department 
of Education, and local education agencies regarding curriculum and instruction; advises and 
makes recommendations to the SBE on implementing the state’s academic content standards; 
and advises the SBE on professional development, pupil assessments, and academic 
accountability systems alignment to the standards.”2! The IQC has been integral in the process 
of translating California’s grade-level content standards into frameworks, and California’s 
Communities of Practice have helped connect the frameworks to teachers and administrators.?? 
Surveys of teachers and administrators suggest strong practitioner support for the standards 
and frameworks. 


18 The Concurrence Committee includes one representatives selected by each of the following: the Regents of the 
University of California, the Board of Trustees of the California State University, the State Board of Education, the 
Governor, the Superintendent of Public Instruction, the Commission on Teacher Credentialing, and the Curriculum 
Development and Supplemental Materials Commission as well as one representative of the California Community 
Colleges selected by the Board of Governors of the California Community Colleges and one representative of an 
independent postsecondary institution selected by the Association of Independent California Colleges and 
Universities (SB 612 2011). 

0 The Instructional Quality Commission was previously named the Curriculum Development and Supplemental 
Materials Commission. It was established in 1927 (California State Board of Education Website: Instructional 
Quality Commission, https://www.cde.ca.gov/be/cc/cd/, accessed on December 7, 2017). 

21 The IQC’s members are primarily appointed by the California Board of Education. Specifically, the State Board of 
Education appoints thirteen members to the IQC, the Governor appoints one member, the Speaker of the 
Assembly appoints one member, and the Senate Rules Committee appoints one member. A State Senator and a 
State Assembly member constitute two additional members of the |QC. There are two main categories of members 
on the IQC: “at least seven should be recognized authorities in specific subject matter fields” and “at least seven 
should be current k-12 classroom teachers, mentors, or both” (California State Board of Education Website: 
Instructional Quality Commission, https://www.cde.ca.gov/be/cc/cd/, accessed on December 7, 2017). 

22 State/Regional Leader 011. 
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Assessment 

California began aligning its assessments to its curricular frameworks in 1993 with the California 
Learning Assessment System (CLAS). The assessments generated considerable controversy after 
they produced high student failure rates, and they were discontinued in 1995.”? California 
started using the California Standardized Testing and Reporting program (STAR) in 1997, then 
switched to the California Assessment of Student Performance and Progress (CASPP) and 
Smarter Balanced assessments to assess student performance, aligned with state content 
standards. 


Enormity of the Current Task 
Though standards-based reform is not new to California, this latest installment is more 


ambitious and coincides with a challenging landscape for the reforms to “take root.” The 
challenges of constructing coherent structures of instructional support prevail not just in 
California, but confront all states in the Common Core era. These challenges include: 


e How to coordinate the actions and products of many independent private sector 
curriculum designers, publishers, and materials providers so that they deal with the 
same knowledge and skills? 


e How to coordinate the actions and products of many independent private and public 
sector teacher education and professional development agencies so that they instruct 
teachers in the same knowledge and skills? 


e How to develop the means to monitor instructional quality, and the means to use the 
knowledge that results to improve instruction? 


e How to coordinate these functions, and the agencies that perform them, so that they 
attend to the same knowledge and skills? 


e How to coordinate these functions, and the agencies that perform them, when the 
school systems that offer instruction are so unequally resourced, and when those 
systems reside in varied racial, ethnic, linguistic, and socio-economic contexts? 


California presents additional challenges to coherence, given its size, diversity, and governing 
structure. It bears remembering that California has over 1,000 school districts: 55 percent are 
classified as small school districts, many of them rural. Yet, these small districts educate only 7 
percent of California’s public school students.”4 The spirit of standards-based reform aspires for 
rigorous, quality instruction for all students, in all schools. Supporting that aspiration collides 
with significant material differences. In California, geography matters: 


23 On lessons from CLAS see Knudson et al. (2015). 

*4 California School Board Association, “Small School Districts,” 

https://www.csba.org/sitecore/OldHome DoNotPublish/Educationlssues/Educationlssues/SmallSchoolDistricts.as 
px Accessed on February 28, 2018. 
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“Implementation is not always as easy in a smaller school. With only 8 
certificated staff, 7 are full time teachers, it is tough to hit all the buttons ina 
high school but still comply with sometimes unrealistic necessities or levels of 
achievement that are expected at, say a school of four thousand with 75 staff. 
There’s diversity in how that educational process is dispensed” (Superintendent 
Interview 043). 


Capacity to put policy into practice is relative: it depends on the size of the task relative to the 
expertise and resources available to those charged with enacting the policy. Sufficient time and 
sufficient expertise matter. Time and staff burdens associated with administrative and 
accountability requirements loomed large in rural superintendents’ concerns. 


“AS a Superintendent of a small district, really [understand] the number of 
demands upon us, the number of roles we play ... At the end of the day, you are 
one or two people trying to fulfill every role ... We have over 10 districts in our 
county with less than 300 students, but we are accountable for everything. Our 
LCAP is the same number of pages, our budget, you name it, everything is the 
same, and you’re asking 1-2 people to do that with quality” (Superintendent 
Interview 066). 


And the challenges facing particular populations of students, such as English Learners and 
students with special needs, amplify the enormity of the task of putting ambitious learning 
standards into practice deeply and equitably. 


“How do we meet the needs of English learners? Because, as a state, we have 
struggled with that ... if we can’t get our brightest minds on a subject matter [as 
a state] to figure it out, how do we expect your average LEA to figure it out?” 
(State/Regional Leader Interview 015) 


Respondents noted that the state has made progress integrating English Language Arts (ELA) 
and English Language Development (ELD) standards, and that instructional materials have 
begun to become more available for students who are learning English or who have special 
needs. 


“TAt first], there wasn't a lot of good materials for the struggling learner or 
English learner ... if you fast forward to now, we're doing better. | don't know if 
we’ve perfected, or we’ve got it right yet” (State/Regional Leader Interview 035). 


We discuss elsewhere in this series (Moffitt et al. 2018) that teachers from schools with higher 
concentrations of poverty or higher concentrations of students who are English Learners were 

significantly fess likely to perceive that their instructional materials are well suited to the needs 
of their students than teachers in low poverty, low-ELL schools. 
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Compounding the geographic challenges facing California, the combination of the 2008 Great 
Recession and scarce financial resources pose additional impediments to deeply and equitably 
putting ambitious academic standards into instructional practice. 


“We are pretty much faced with financial turmoil ... Curriculum is not even on 
our radar when it comes to a sense of priority, it’s the budget” (Superintendent 
Interview 027). 


Figure 3.1: Top Superintendent Concerns 
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Fig 3.1 Source: structured interviews conducted with a stratified random sample of California school district superintendents, 
conducted between June and October 2017. See the Appendix for selection information and interview protocol. 


As Figure 3.1 suggests, our interviews of a stratified random sample of district superintendents 
revealed that fiscal and personnel matters dominated superintendents’ list of priorities. Indeed, 
professional learning opportunities rarely appeared on superintendents’ “radars.” 


Economic constraints can render professional development particularly vulnerable. As the 
leader observed below, “when the budget is cut, anything that's not personnel, just goes out 
the door,” especially for districts that did not have instructional support already stitched into 
district culture and priorities. 


“{Common Core implementation] came on the heels of the big recession, so just 
implementing the first two subject areas, reading language arts and math, was a 
real struggle for most of the districts because they didn't have the wherewithal 
to do...professional development if that was even in their culture in the first 
place. It's an interesting phenomenon because the high-functioning districts in 
this state, they would not even think of dropping professional development as a 
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major focus because they realize that's their lifeblood of having their staff 
understand what they're supposed to be doing, but in the places, that don't have 
any of that culture, it's just—it's a throwaway. When the budget is cut, anything 
that's not personnel, just goes ... out the door” (State/Regional Leader Interview 
010). 


Districts that strive to coordinate the elements of instruction undertake extraordinary efforts to 
do so. Consider the example below, in which the superintendent describes curating the vast 
terrain of instructional materials and assessing their effectiveness. 


“We just adopted the language arts program for next year. What we do is we 
utilize the expertise of our teachers. We brought a group together over the 
course of 24 months, researched all the materials that were available and 
aligned to standards. It was really driven by teachers, and they got feedback 
from their colleagues. So that was the process we utilized to ensure that the 
practitioners are getting the opportunity to preview and test drive materials that 
are out there to determine if they’ re effective ... They [the teachers] actually 
built a process whereby data was collected from teachers that were piloting 
materials, and they brought back and analyzed the results they got, quantitative 
and qualitative results from classroom teachers. Piloting went on for about 18 
months, so it was very comprehensive” (Superintendent Interview 159). 


As we mentioned earlier, one way to solve the technical, organizational, and political puzzles 
we posed at the outset involves building a school system with coherent instruction and 
instructional guidance, on the assumption that the school system is the chief unit of action, and 
that coherence can best and most durably be built at that level. Some LEAs have made 
significant progress on several of the puzzles by building such internally coherent systems of 
instruction and instructional guidance. Doing so, however, requires extraordinary effort and 
expertise on the part of the LEA. 


These districts’ ambitious efforts to encourage teachers’ deep engagement with ambitious 
standards and aligned elements of instructional support underscore the overarching problem of 
curating the current terrain. This places the responsibility for navigating the expanded terrain of 
new materials on the shoulders of teachers, teacher leaders, and school leaders. 


“A big area that is missing is curation: trying to figure out how to teach 
mathematics curriculum, where do we go for good information? People 
shouldn’t have to reinvent the wheel ... People go to Pinterest and get 
overwhelmed” (State/Regional Leader Interview 011). 


Some states, like Louisiana, curate the terrain of materials more than California. In California’s 
vast and varied terrain, some LEAs are creating coherent systems themselves. For others, 
“curriculum is not even on our radar,” because of overarching financial pressures 
(Superintendent Interview 027). 
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Given the enormity of the task, it bears remembering that California teachers’ modal response 
is “improved a little” to a series of questions about key components of instruction: alignment 
between standards and materials, alignment between district professional learning 
opportunities and teacher needs, the quality of instructional materials, the quality of 
professional learning opportunities, and the quality of school-level learning communities. Given 
the enormity of the task, “improved a little” can be understood as a remarkable 
accomplishment. California’s vast, varied, and decentralized terrain invites revisiting the idea of 
complementarities and positive-sum organizational arrangements.”° Centralized support— 
which differs substantively from centralized oversight—can support decentralized decision- 
making, upon which LCFF and Common Core depend. At the state level, what sources of 
capacity to support instructional improvement operate in California? How are they distributed? 
How does California’s capacity compare with other states? How might capacity be 
strengthened? We turn to these questions now. 


3.2 Limited CDE Resources Relative to the Enormity of the Task 

California’s embrace of Common Core State Standards coincided with material constraints on 
the state department’s ability to support putting those standards into practice. Figure 3.2 
below depicts the overall trend-line for state funded positions in K-12 education. This reflects 
the growth in positions overall since the 1990s, along with recent declines. 


Figure 3.2: Total State-Funded Positions in K-12 Education 
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Figure 3.2 Source: Governor's Proposed Budget 2007-2008 to 2017-2018, http://www.ebudget.ca.gov/, accessed February 23, 
2018; California State Legislative Analyst’s Office, www.lao.ca.gov/sections/econ_fiscal/Historical_ Personnel _Years.xls, 
accessed on February 23, 2018. 


Focusing more specifically on the CDE, a 2014 review of the CDE similarly reported more CDE 
staff in 2014 relative to the 1990s. However, the number of staff declined by 119 positions 
between 2008 and 2014. Moreover, 39 percent of the staff reductions that occurred between 


5 For a helpful discussion of complementary organizational arrangements, see Cammett and MacLean (2014). 
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2008 and 2014 were incurred in areas of CDE that connect with instructional support: 
Professional Learning Support, Assessment Development and Administration, and Improvement 
and Accountability (Taylor 2014, 14).2° Part of the explanation for these staff reductions comes 
from California’s shift away from state categorical funding and toward the Local Control 
Funding Formula (LCFF). Some of the eliminated positions had been responsible for the 
oversight and monitoring of state categorical grants. Nevertheless, California embarked on 
simultaneous, major reform efforts—adoption of the Common Core and the shift to LCFF—at 
the same time that it downsized its state agency, charged to “compile and disseminate data on 
districts, schools, staff, and students; oversee development of curriculum frameworks, 
standardized student assessments, instructional materials, and school facilities standards” and 
to “oversee county offices of education” (LAO 2018, 5). 


How does California’s investment in state department staffing compare with other states? 
Figure 3.3 below depicts state department staffing levels per 1,000 public school students in the 
state in 2017. The nine other comparison states were selected to reflect variation in state size, 
region, and partisan affiliation. California’s relatively low rate of state-level staffing relative to 
student enrollment is commensurate with other big states, like Texas and Florida. 


Figure 3.3: 2017 Department of Education Staff Per Students 
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Figure 3.3 Source: State department of education websites for all available offices. Please see the Appendix for source 
information. Note that this figure uses pupils based on the National Center for Education Statistics Common Core of Data 2015- 
2016 (includes adult education and pre-k students), https://nces.ed.gov/ccd/stnfis.asp, accessed on February 20, 2018. 


Staff Per 1,000 Students 


Compared nationally and over time, however, California has significantly fewer state education 
department employees, relative to the size of its student population. Table 3.1 offers a simple 


26 Specifically, the LAO report found that, of the 119 positions eliminated, 25 were in Professional Learning 
Support, 11 came from Assessment Development, and 10 came from Improvement and Accountability (Taylor 
2014, 14). 
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difference in means between California and the rest of the states for the 6-year period from 
2010-2015.?” 


Table 3.1: T-Test, California Compared with all States’ Bureaucratic Capacity, 2010-2015 


Mean Employees per 1000 students 


Group Obs Mean Std. Err. Std. Dev. 
All States 294 2.363 0.073 1.254 
California 6 0.691 0.005 0.012 
Difference 1.672 0.513 

t= 3.261 


Pr(T >t) = 0.001 
Table 3.1 Source: U.S. Census Annual Survey of Public Employment and Payroll, mean number of employees per 1,000 students 
from 2000-2015, https://www.census.gov/programs-surveys/apes/data/tables.html, accessed on January 16 and 17, 2018. 
Note that this figure uses pupils based on the National Center for Education Statistics Common Core of Data (includes adult 
education and pre-k students), https://nces.ed.gov/ccd/stnfis.asp, accessed on February 20, 2018. 


Is the CDE’s capacity sufficient? Again, capacity is relative to the task at hand: it is not an 
absolute. Thus, staffing levels overall are only modest proxies, at best, of capacity to support 
ambitious instruction aligned with California’s content standards. The 2014 review of the CDE 
concluded “the CDE’s overall staffing level is reasonably well aligned with its existing 
responsibilities” but that it has “limited capacity to absorb new workload” (Taylor 2014, 3). The 
LAO report went on to recommend that “when the state tasks the CDE with notable new 
requirements—either through the annual budget act or through other legislation—the 
Legislature provide the department with additional positions and funding to carry them out” 
(Taylor 2014, 3). Though the adoption of ambitious content standards and the implementation 
of LCFF did not come with specific “notable new requirements” for the CDE, the policies overall 
place enormous new expectations on all levels of the system to change in significant ways. One 
could interpret both as imposing new workloads in the CDE. In the words of one state leader: 


“| think CDE probably has less capacity now [than in the 1980s] but the 
challenges are even greater [now] than they were at that time” (State/Regional 
Leader Interview 010). 


Yet, is instructional support the responsibility of the CDE? The LAO report noted at its outset: 
“Given the size of the state, number of LEAs, and diversity among LEAs, the state typically relies 
on county offices of education (COEs)—not CDE—to provide direct assistance and specific 
advice to LEAs on how they can improve their educational programs” (Taylor 2014). The 
California Collaborative for Educational Excellence is also responsible for select interventions. 
As the agency charged with administering federal and state policy, however, the CDE has at 
least formal responsibility to help put California’s ambitious grade-level standards into 
instructional practice deeply and equitably. 


27 We offer this time span because the data are available for all states during those years. Data for some states are 
not yet available for 2016 and 2017. 
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Figure 3.4: Superintendents’ Sources of Materials 
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Fig 3.4 Source: structured interviews conducted with a stratified random sample of California school district superintendents, 
conducted between June and October 2017. See the Appendix for selection information and interview protocol. The data 
presented is categorized from an open-ended question: “Where do you receive information about standards-aligned curriculum 
and instructional materials?” (Question 3.2.1, Superintendent Interview Protocol — see Methods and Data Appendix). 
Respondents often listed multiple sources. 


Consistent with the view that county offices of education are the primary sources of 
instructional support for districts, our interviews of 91 district superintendents, drawn from a 
stratified, random sample found that 55 percent of the superintendents interviewed look to 
county offices for information on instructional materials.2° Yet, Figure 3.4 also reveals that 
many districts look to the state for support. Our interviews revealed that 43 percent of the 
respondents looked to the state for information on instructional materials. In districts with high 
concentrations of ELL students, 52 percent of superintendents reported looking to the state for 
information on instructional materials.”° 


Our interviews found a similar trend for sources of information on LCFF and LCAP. Again, our 
respondents were most likely to look to county offices for information on LCFF and LCAP 
implementation (Figure 3.5). Sixty-nine percent of our respondents also reported using 
information from the state about LCFF and LCAP implementation. Eighty-two percent of 
respondents from high ELL districts reported using information from the state about LCFF and 
LCAP implementation. 


28 A stratified random sample of 205 California school district superintendents were asked to participate in a 30- 
minute structured interview to systematically retrieve their views on policies and conditions affecting their 
districts. A total of 91 district superintendents agreed to participate and were successfully interviewed. Interview 
topics included: the implementation of educational standards, the implementation of the Local Control 
Accountability Plans, school finance, staffing needs, and data use. Please see the Appendix for selection 
information and interview protocol. 

22 29 of our 91 superintendents are leaders of districts with high concentrations of English learner students. 
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Figure 3.5: Superintendents’ Sources of LCAP/LCFF Guidance 
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Fig 3.5 Source: structured interviews conducted with a stratified random sample of California school district superintendents, 
conducted between June and October 2017. See the Appendix for selection information and interview protocol. The data 
presented is categorized from the list question: “From the following list, where do you receive guidance and support on LCAP and 
LCFF” (Question 4.4, Superintendent Interview Protocol — see Methods and Data Appendix). Respondents were invited to answer 
yes/no to a list of multiple sources, and to supplement with others. 


Figure 3.6 reveals that few superintendents reported receiving information from the state 
about professional learning opportunities and providers. Here, other superintendents presents 
as the modal source of information. 


Figure 3.6: Superintendents’ Source of 
Professional Development Guidance 
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Fig 3.6 Source: structured interviews conducted with a stratified random sample of California school district superintendents, 
conducted between June and October 2017. See the Appendix for selection information and interview protocol. The data 
presented is categorized from an open-ended question: “How do you learn about providers of professional development and 
their quality?” (Question 3.3.1.1.3, Superintendent Interview Protocol — see Methods and Data Appendix). Respondents often 
listed multiple sources. 
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With superintendents looking to and receiving information from the state on materials and 
LCFF implementation, is the CDE well-positioned to provide that and other forms of information 
and support for instructional practice? Here, several challenges appear. 


3.3 Limited In-House Content Matter Specialists 

Our semi-structured interviews with state and regional leaders revealed strong support for CDE 
leadership and the direction it has taken the agency with respect to orienting the agency 
toward creating partnerships to encourage instructional improvement and continuous 
improvement. Concerns emerged, however, over the relative dearth of CDE’s content-specific 
or subject-matter expertise. The department—for part of the 1980s—had previously been 
organized around subject areas (State/Regional Leader Interview 001 and 011). That is no 
longer the case. Along with the different reorganization priorities has come a reduction in CDE 
in-house content expertise. As Timar and Carter reported, “According to Bill Honig, the CDE had 
three math specialists for 10,000 schools in the 1990s. Today it has none” (B. Honig personal 
communication, 2013 reported in Timar and Carter 2017, 11). 


One recent source of reduction in the CDE’s content matter expertise arose with the 
elimination of state categorical funding. 


“The content expertise is no longer present because we don't have all the 
categoricals that were all content-oriented around professional learning and 
development” (State/Regional Leader Interview 012). 


A long-standing impediment to attracting and retaining content matter specialists in the 
department arises from the structure of departmental funding. The 2014 LAO review of the CDE 
revealed that 68 percent of CDE funding comes from federal funds (Taylor 2014, 3). State 
departments’ dependence on federal sources of funding infuses federal priorities—such as 
compliance—into state agencies. Federal funding strings can also be extremely time 
consuming. After meeting federal grant monitoring requirements, state staff can have little 
time left for other kinds of work.?° 


“We're seeing these state agencies become auxiliary federal agencies ... what 
that's looked like in our California Department of Education is ... what formerly 
were deep content managers have turned into grant managers and have turned 
into more of the facilitators to the field of ... federal monitoring” (State/Regional 
Leader Interview 012). 


Without state sources of flexible funding for the CDE, the agency has few opportunities to 
provide instructional support to the counties and districts that seek its help. 


30 State/Regional Leader Interview 033. 
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“The department staff is just more or less funded on federal funds and is doing 
federal activities ... If we had some room for them to exercise leadership in 
things like professional development, for example, or in curriculum, where they 
could offer help and assistance to the community, that would be a positive thing 
... The situation we’re in is that there’s no money to do that, and there’s no 
ability to provide leadership for that” (State/Regional Leader Interview 035). 


One way that the state overall—the CDE and the SBE—taps content experts is in the framework 
and instructional material adoption process. This offers a vehicle to draw expertise from the 
field into the agency and supports the content and quality of the agency’s products. 


“I’ve certainly come to have a great deal of respect for all of their work around 
the frameworks, and the instructional material, the adoption process ... they use 
teachers, to an amazing extent, in all their curriculum committees, and their 
framework-adoption committees” (State/Regional Leader Interview 037). 


Another way to tap subject-matter expertise, with the potential to provide support to counties 
and districts in addition to supporting the quality of CDE products, is through an exchange 
model, which we discuss further in the conclusion to this section. 


3.4 Attraction and Retention of Quality Staff 

Our interviews of state and regional leaders frequently pointed to relatively low pay among CDE 
staff as a potential factor in the agency’s challenge attracting and retaining staff members in 
general, and staff who are content matter specialists in particular: 


“| feel bad for the CDE. | don't know if it’s a funding issue or ... what the issue is, 
but they have a high turnover ... it seems like the level and depth of expertise 
and experience they’re not able to retain, attract and retain staff ... and they 
move around pretty rapidly and so, | don't know if that’s because of a salary 
schedule thing or what it is. They need to be able to hire and retain highly skilled 
experts” (State/Regional Leader Interview 004). 


Several sources of data confirm this perception. Figure 3.7 compares CDE salaries with other 
states. California state department salaries are indistinguishable from Florida and Tennessee, 
but significantly lower than those in Texas, Massachusetts, and Minnesota. These salaries are 
not adjusted for differences in cost of living across the states. 
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Figure 3.7: Median Department of Education Salaries Across States, 2016 
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Moreover, the CDE competes with other education agencies in the state. Figure 3.8 compares 
mean salaries of CDE management with district leaders’ salaries, for the 10 districts with 


highest enrollment in the state. 


Figure 3.7 Full source information appears in the Appendix. 


Figure 3.8: Average Salary Top 5 Highest Paid Officials in 10 Highest 
Enrollment Districts and Sacramento Compared to Average Salary C.E.A.s in 
the CDE, 2015 
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Figure 3.8 Source: Transparent California, https://transparentcalifornia.com/agencies/salaries/school-districts/, 
accessed on January 8, 2018. The Sacramento Bee, http://www.sacbee.com/site-services/databases/state- 
pay/article2642161.html, accessed on January 12, 2018. California Department of Education, 
http://web.archive.org/web/20160511021920/http://www.cde.ca.gov/ds/sd/cb/ceflargesmalldist.asp, accessed on 
January 19, 2018. Note: Sacramento is not one of the 10 districts with highest enrollment but is included. One C.E.A. 
(Career Executive Assignment) salary was omitted due to it being an outlier. C.E.A.s do not include Chief Counsel 
C.E.A.s. 
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The mean of CDE managers’ salaries is significantly lower than the mean of the top-5 highest 
paid district employees, in the 10 districts with the highest enrollment in the state.?+ 


Table 3.2: T-Test, Average C.E.A. Salaries in CDE Compared with Top 5 Highest Paid District Employee 
Salaries in 10 Highest Enrollment Districts and Sacramento, 2015 


Groups Obs Mean Standard Error St. Dev. 
Districts 55 $218,773 $6,453 $47,859 
CDE 26 $113,803 $5,983 $30,508 
Difference $104,969 

t= 10.226 


Pr(T >t) = 0.000 


Table 3.2 Source: Transparent California, https://transparentcalifornia.com/agencies/salaries/school-districts/, accessed on 
January 8, 2018. The Sacramento Bee, http://www.sacbee.com/site-services/databases/state-pay/article2642161.html, 
accessed on January 12, 2018. California Department of Education, 
http://web.archive.org/web/20160511021920/http://www.cde.ca.gov/ds/sd/cb/ceflargesmalldist.asp, accessed on January 19, 
2018. Note: Sacramento is not one of the 10 districts with highest enrollment but is included. Please note that one C.E.A salary 
was omitted due to it being an outlier. C.E.A.s do not include Chief Counsel C.E.A.s. 


More variation appears at the county level, depending on the county, as Figure 3.9 suggests. 
Yet, here again, the mean salary of CDE management (C.E.A.s) is significantly lower than the 
mean salary of top officials at the county level. 


Table 3.3: T-Test, Average C.E.A. Salaries in CDE Compared with Top 5 Highest Paid County Employee 
Salaries in 20 Highest Enrollment Counties, 2015 


Groups Obs Mean Standard Error St. Dev. 
Counties 100 $193,755 $3,530 $35,304 
CDE 26 $113,803 $5,983 $30,508 
Differences $79,951 

t= 10.560 

Pr(T >t) = 0.000 


Sources: Transparent California, https://transparentcalifornia.com/agencies/salaries/school-districts/, accessed on January 8, 
2018 and February 26, 2018. The Sacramento Bee, http://www.sacbee.com/site-services/databases/state- 
pay/article2642161.html, accessed on January 12, 2018. California Department of Education, 
https://dq.cde.ca.gov/dataquest/dqcensus/EnrEthGrd.aspx?cds=01&agglevel=county&year=2014-15, accessed on January 13, 
2018. C.E.A.s do not include Chief Counsel C.E.A.s. 


31 We focused on districts and counties with the highest enrollment as our comparison group, because interviews 
suggested they were more likely to be a source of competition for the CDE that smaller districts and counties. 
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Figure 3.9: Average Salary Top 5 Highest Paid Officials in 20 Highest Enrollment Counties 
Compared to Average Salary C.E.A.s in the CDE, 2015 
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Figure 3.9 Sources: Transparent California, https://transparentcalifornia.com/agencies/salaries/school-districts/, accessed on 
January 8, 2018 and February 26, 2018. The Sacramento Bee, http://www.sacbee.com/site-services/databases/state- 
pay/article2642161.html, accessed on January 12, 2018. California Department of Education, 
https://dq.cde.ca.gov/dataquest/dqcensus/EnrEthGrd.aspx?cds=01&agglevel=county&year=2014-15, accessed on January 13, 
2018. C.E.A.s do not include Chief Counsel C.E.A.s. 32 


Why might lower salaries matter to CDE’s ability to support the ambitious goals of standards- 
aligned instructional improvement? In addition to the challenge lower salaries may pose to 
attracting content experts, our interviews pointed to ways in which turnover can impede 
relationships of support with county and district staff. With stable relationships 


“... you develop a relationship and you’re much more likely to ... say, ‘Hi,’ and 
you’re struggling with this particular issue and be vulnerable and be willing to 
show the things that you’re struggling with and at that point then seek together 
expertise” (State/Regional Leader Interview 004). 


Interviews also pointed to situations when the county or district person asking for state 
help had more content knowledge than the state staffer: 


“Sometimes, we’re actually ... the person who ... has more experience or 
knowledge of those areas ... They’re [the district person] actually training the 


32 These are the county offices of education. Not all county offices of education are named ‘county offices of 
education.’ Other names include: Kern County Superintendent of Schools, Orange County Department of 
Education, and San Bernardino County Superintendent of Schools. San Francisco is the 22" county with the highest 
enrollment. San Bernardino’s average reflects the base pay. To obtain 2015 salary data, we averaged the 2014 and 
2016 salary data for the Mono County Office of Education and the San Mateo County Office of Education. 
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CDE person ... on their program that they’re there to monitor and that’s not 
comfortable for anybody” (State/Regional Leader Interview 004). 


As this last comment illustrates, when contact occurs between the CDE and counties or 
districts, the topic of that contact is often monitoring and compliance. 


3.5 Workload and Compliance Orientation 

The CDE is in the midst of a significant reorganization, aimed at becoming more of a support 
organization. Departmental leaders have articulated a clear vision and have taken concrete 
organizational steps toward reorienting the agency to become a structure of support.?? 
Challenging this organizational shift is departmental funding that remains tightly connected to 
federal funding. Reliance on federal funding poses two related kinds of problems: workload and 
lingering “compliance” oriented perspectives among staff.*4 


The flow of federal funds to state departments of education has, over the past fifty years, been 
an important source of expanding state departments budgets and staffing levels (Murphy 
1974). Yet, along with federal funds come monitoring, oversight, and reporting requirements. 
State obligations to satisfy federal requirements can both consume staff time and set the terms 
of staff priorities. As we noted above, the CDE’s rate of state-level staffing per K-12 pupil 
enrollment is relatively low compared with other states, and resembles staffing rates in other 
large states such as Texas and Florida. Yet, relatively more federal funds pass through the CDE 
than through other states’ agencies. Administering federal grants and fulfilling the reporting 
requirements that come with federal grants can be enormously time consuming and can crowd 
out other work (State/Regional Leader Interview 033). These conditions risk exacerbating 
workload in the CDE. 


33 Torlakson, Tom and Glen Price. (2016, November 21). Towards a Learning State: California’s Journey Toward 
Continuous Improvement [Web blog post], 

from http://blogs.edweek.org/edweek/on_california/2016/11/towards a learning state californias journey tow 
ard_continuous_improvement.html?cmp=SOC-SHR-FB;, accessed on November 11, 2017;; Superintendent's 
Accountability and Continuous Improvement Task Force, California Department of Education (2016), Preparing All 
Students for College, Career, Life, and Leadership in the 21st Century, retrieved 

from http://www.cde.ca.gov/ta/ac/ar/account-report-2016.asp; California Department of Education, Blueprint 2.0 
Planning Team (2015). A Blueprint for Great Schools, Version 2.0., retrieved 

from http://www.cde.ca.gov/eo/in/bp/documents/yr15bp0720.pdf; Torlakson, Tom and Glen Price. (2017, April 
10). Calif. Ed Department Reorganizes Into Teams [Web blog post], 

from http://blogs.edweek.org/edweek/on_california/2017/04/calif_ ed department_reorganizes into teams.html 
?cmp=SOC-SHR-FB;, accessed on November 11, 2017; Torlakson, Tom and Glen Price. (2017, February 22). Creating 
a Learning Community at California’s Education Department [Web blog post], 

from http://blogs.edweek.org/edweek/on_california/2017/02/creating a learning community at californias ed 


ucation_department.html?cmp=SOC-SHR-FB., accessed on November 11, 2017. 
34 On other states’ “compliance mindset” see Jochim and Murphy (2013). 
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Figure 3.10: Revenue from Federal Sources Distributed Through the State for Public 
Elementary-Secondary School Systems by Enrollment in Constant Dollars, 2002-2015 
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Figure 3.10 Source: U.S. Census Annual Survey of School System Finance Tables, https://www.census.gov/programs- 
surveys/school-finances/data/tables.html, accessed on April 2, 2018 and on January 4, 2018. In thousands of dollars. 


Figure 3.11: Revenue from Federal Sources Distributed Through the State for Public 
Elementary-Secondary School Systems by Employees in Constant Dollars, 2002-2015 
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Figure 3.11 Source: U.S. Census Annual Survey of School System Finance Tables, https://www.census.gov/programs- 


surveys/school-finances/data/tables.html, accessed on April 2, 2018; U.S. Census Annual Survey of Public Employment and 
Payroll, https://www.census.gov/programs-surveys/apes/data/tables.html, accessed on January 16, 2018. In thousands of 


dollars. 
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Figure 3.10 plots California and other states’ federal funding distributed through states, in 
constant dollars, relative to student enrollment. For all years with the exception of 2010 and 
2011, California distributes more federal funds that the comparison states, including the large 
comparison states. 


Figure 3.11 plots California and other states’ federal funding distributed through states, in 
constant dollars, relative to state staff levels. For all years, California distributes more federal 
funds that the comparison states, including the large comparison states. 


These conditions also risk exacerbating goal displacement, where federal priorities supplant 
state priorities. When the department struggles to compete with counties and districts for 
quality staff, given the department’s salary constraints, this can also interfere with stable 
relationships, consistent messages, and expertise. The new era of LCFF expresses the explicit 
state priority of local control and configuring state structures to support localities in their LCFF 
implementation: to move from centralized state oversight to coherent state support. 


“In the LCFF era, we have moved from categorical programs of legislative design 
to working with counties and districts in helping them design programs for their 
students” (State/Regional Leader Interview 001). 


Perceptions of inconsistent CDE messaging continue to emerge from the field, however. 


“So, the CDE is definitely in a restructure and we appreciate that. They’re 
aligning a lot of their work. What we’re still hearing in the field ... [is] we’re not 
quite there yet in terms of coordinating and creating a common message across 
divisions ... [especially for] LCAP ... [and] in terms of addressing particularly 
students with disabilities” (State/Regional Leader Interview 004). 


Frustrations with inconsistent messaging emerged from our superintendent interviews as well, 
along with calls to: 


“address the disconnect between what elected officials say and what the state 
staff of the department of education do. They’re very compliance minded, 
they’re not quite [as] on board with the new LCFF and LCAP as staff members, 
and those are the ones we interact with daily” (Superintendent Interview 059). 


While recognizing the strength of the CDE’s current leadership and supporting its effort at 
reorganization, concern emerged over the future and whether the recent changes would have a 
chance to embed and take root. 


“In the absence of good leadership, and | think we’re in a place right now, where 
we do have good leadership in the department, then it’s easy for that federal 
compliance thing to take over, and it just sort of swamp all this other work that 
they do” (State/Regional Leader Interview 037). 
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This expresses the dual problem associated with federal funding: it can displace state goals 
(“that federal compliance thing” can “take over”) and it can create a burdensome workload for 
state employees (that “swamp all this other work that they do”). Figures 3.10 and 3.11 above 
suggest these conditions are particularly pronounced in California. 


3.6 Opportunities to Learn: Building Expertise 

We mentioned at the outset that we take several things as given: the constitutional structure of 
California’s education governance; financial constraints; the recent move to decentralization. In 
addition to recent policy shifts toward decentralization, California’s size, scope, and diversity 
render it challenging for the CDE to play a direct role in instructional support: 


“| think the CDE is a group that is extremely important to the entire system. 
When it comes to the on the ground standards work ... the CDE in our state will 
never be able to do what the Arkansas Department of Education can do in their 
state, meaning that in Arkansas they have a team of folks at the Department of 
Ed that will go out across their state and provide professional development. 
That’s part of their mission and structure. Arkansas is a state that if you drive 
three hours from Little Rock, you could hit every corner of the state. In our state 
it’s not possible. We have to accept that, and we have to also acknowledge that 
the CDE should grow to be a support organization, but ... it’s always gonna be 
indirect support by facilitating the subject matter projects, by facilitating the 
county office of education, by facilitating non-profit providers to play that role of 
increasing the quality of support out there...” (State/Regional Leader Interview 
015). 


What might a “support organization” that provides “indirect support” entail? How might a 
central state support decentralized practice, with the state, county, district, and non- 
governmental agencies working as complementaries? The county offices of education, the 
California Subject Matter Projects, and the vast terrain of non-profit providers in the state play 
active, direct roles in technical support, with the CDE providing indirect support. Yet, the 
technical, organizational, and political puzzles we posed put coordination front and center. 
Even with counties and non-governmental entities providing the lion’s share of direct technical 
assistance, important matters of coordination and need for the central state expertise to 
support decentralized decision-making remain. 


Coordination and central-state support require skills, know-how, and values. As noted at the 
opening, “Common Core Implementation requires CDE staff to have more expertise than in past 
standards implementation efforts” (State/Regional Leader Interview 001). What might be done 
to augment the technical capacity of the CDE as a key structure of support for instructional 
improvement? We examine two opportunities: revisit pay structures to attract staff expertise 
to CDE and draw on local expertise to continuously infuse the CDE with frontline knowledge. 


Other agencies in California facing labor market pay disparities provide opportunities to learn 
how to address retaining and attracting quality staff. 
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Figure 3.12: Average Salary Top 5 Highest Paid Employees in 10 Highest Enrollment 
Counties Compared to Average Salary C.E.As in Water Resources, 2016 
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Figure 3.12 Source: Government Compensation in California, publicpay.ca.gov accessed on April 16, 2018. California 
Department of Education, 
http://web.archive.org/web/20160511021920/http://www.cde.ca.gov/ds/sd/cb/ceflargesmalldist.asp, accessed 
January 19, 2018. 35 Please note that one outlier C.E.A salary was omitted. 


Disparities between state agencies’ pay and the payscales in other public-sector labor market 
options are not unique to education. As California’s Department of Water Resources 
established in 2011, “DWR’s payscales and the median of those of other public sector-utilities 
now exceed 30 percent, contributing to a severe recruitment and retention crisis within its 
Operations and Maintenance Workforce” (DWR 2011). The 2013 addendum to the union’s 
contract increased salaries between 17 to 37 percent for a range of job classifications at the 
Department of Water Resources. 


For comparability with our education analysis, which compares state with county salaries, 
Figure 3.12 compares state-level water resource salaries with county-level salaries. Table 3.4 
presents a difference in means for 2016, several years after the pay adjustment. The difference 
is indistinguishable from 0 at conventional levels of statistical significance.*° Salary adjustments 


35 The departments included are Alameda- Maintenance and Operations; Fresno- Public Works and Planning; Kern- 
Public Works and Planning; Los Angeles- Public Works; Orange- Public Works; Riverside- Facilities Management; 
Sacramento- Water Resources; San Bernardino- Public Works; San Diego- General Public Works; Santa Clara- 
Facilities. 

36 This does not, of course, mean that there is no difference. Rather, the level of uncertainty is sufficiently high to 
preclude discerning a difference. 
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that brought state salaries into closer alignment with local salaries have also occurred for state 
prison nurses and for engineers (Ortiz, 2013). 


Table 3.4: T-Test, Average C.E.A. Salaries in Water Resources Compared with Top 5 Highest Paid 
County Employee Salaries in 10 Highest Enrollment Counties, 2016 *” 


Groups Obs Mean Standard Error St. Dev. 
Counties 50 $168,329 $7,149 $50,552 
State 21 $154,924 $12,567 $57,590 
Difference $13,404 

t= 0.978 


Pr(T >t) = 0.166 


Table 3.4 Source: Government Compensation in California, publicpay.ca.gov accessed on April 16"", 2018. California Department of Education, 
http://web.archive.org/web/20160511021920/http://www.cde.ca.gov/ds/sd/cb/ceflargesmalldist.asp, accessed on January 19, 2018. 


Adjusting payscales provide an approach to building expertise in the CDE’s ongoing staff. An 
additional approach taps frontline expertise through exchange models. Concern over the CDE’s 
relative dearth of in-house content matter expertise arose as a steady theme in our interviews. 
This concern is not unique to the CDE, but part of a larger trend nationally toward “hollowing 
out” government agencies (Agranoff and McGuire 1998; Milward and Provan 2000). Part of the 
challenge lies in constructing an approach where expertise is accessible, up-to-date, flexible, 
and economically efficient. Adding civil servant positions does not necessarily satisfy these 
conditions. Other countries and other government agencies (Mills and Selin 2017) in United 
States administer exchange programs. 


As state and regional leaders noted in our interviews, an infrastructure existed within California 
to enable rotations between the CDE and other governmental entities (counties or districts). 


"It would be great to have a robust program that would support consistently 
bringing people from the field into the department and department people to 
the field. It would be an exchange-type model. It could help the Department and 
its employees maintain a fresh perspective while ensuring better connection 
between policy and local practice" (State/Regional Leader Interview 001). 


An exchange program offers one option to counter the depletion of content-matter specialists 
housed within the CDE. Part of the promise of an exchange program lies in the content matter 
expertise district or county instructional leaders could provide to the development of state 
products. Part of the promise also lies in the ways in which these instructional leaders could 
provide state-wide, as opposed to only regional, technical support. 


37 The departments included are Alameda- Maintenance and Operations; Fresno- Public Works and Planning; Kern- 
Public Works and Planning; Los Angeles- Public Works; Orange- Public Works; Riverside- Facilities Management; 
Sacramento- Water Resources; San Bernardino- Public Works; San Diego- General Public Works; Santa Clara- 
Facilities. One C.E.A salary was omitted due to it being an outlier. 
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“The CDE staff is stretched thin and more subject matter experts are needed. It 
would be helpful to have more staff with recent experience in local school 
districts” (State/Regional Leader Interview 001). 


Bringing instructional leaders into the state, this implies, could both shape state policy and 
provide guidance to districts and counties looking for help.?® Recall that our interviews of 
superintendents suggest that many do look to the state for help already for materials and for 
LCFF/LCAP guidance. 


This approach would need to find a way to reconcile the pay-differential between state and 
district/county staff who are paid more. 


“| think when the Department of Education is able to do good work it is because 
they have more staff with experience in the field and schools. Right now, they 
have more experienced school and district staff. The pay is not great, district and 
county positions often pay more. If CDE can use a visiting educator structure 
when hiring, people would not have to take a $20,000, $30,000 pay cut” 
(State/Regional Leader Interview 006). 


An exchange program between the CDE and county or district agencies would stand in contrast 
to field-level engagement that focuses on monitoring. Could an exchange program be put into 
practice? Pay structures and contractual agreements currently pose a potential structural 
challenge to bringing county or district expertise into the department. While an Education 
Program Consultant program exists, collective bargaining agreements currently discourage 
bringing visiting educators into the consultant positions. 


We noted that the coordination puzzles that coherent instructional support raises are not 
merely organizational challenges: they are also technical and political. They do, however, raise 
organizational challenges, which we turn to now. 


38 We are referring here to instructional leaders intentionally, recognizing the difference between administrators 
and instructional leaders. 
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4.0 ORGANIZATIONAL CAPACITY 
“That’s just the problem. There are so many resources out there around any 
given topic and they’re not curated. They’re not organized. They’re not bundled 
and pulled together... put yourself in the place of a site principal. So where do 
you go to get your Cliff Notes and have it all pulled together for you? That 
doesn’t exist” (State/Regional Leader Interview 029). 


Putting California’s ambitious grade-level content standards into practice deeply and equitably, 
and aligning all parts of the system requires not just technical know-how. It also depends on the 
organizational capacity to coordinate elements in California’s vast and varied terrain. As we 
noted earlier, the challenges of constructing coherent structures of instructional support 
transcend California but also appear in California. 


We focus here on two forms of organizational capacity of California’s structures for 
instructional support. First, we look to the structural placement and division of [position-based] 
authority and responsibilities, and the ways in which this terrain supports and impedes 
collaboration and coherent action to promote instructional improvement. Second, we look to 
the relationships between constituent and collaborative units, with the same question in mind. 
Several overarching themes emerge from the data and interviews: 


e Praise for the Brown administration’s ability to navigate the structural divides inherent 
in the design of state-level agencies combined with concerns about the durability of the 
gains in cross-agency collaboration. 


e The vast complexity of California’s networked terrain of instructional support, and 
concern over its implications for incomplete and insufficient coverage and curation; and 
the ways this might exacerbate disparities. 


e Cultivating attentiveness to content and the quality of content in a context that 
emphasizes process and connectivity separately from content. 


4.1 State-Level Structural Divisions 

The past eight years have yielded progress toward cross-agency collaboration. However, deep 
structural divisions render collaboration dependent on current leadership rather than 
embedded in the structures themselves. Many prior reports on California governance have 
highlighted the fragmented governing conditions that confront the terrain of state-level 
structures connected to instructional support (Brewer and Smith 2007). 


“One of the governance problems in the CDE is that we have a state board that’s 
making all the policy, and the state superintendent who has no power, but he or 
she is in charge of the CDE ... if you want the Department of Education to 
function effectively, they should be guided by the policies of the state board, or 
there shouldn’t be a state board, and they’d be guided by the policies of the 
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superintendent. You can’t do this hybrid thing that we have right now” 


(State/Regional Leader Interview 035). 


States have different education governance structures and these differences may contribute to 
or discourage collaboration. One of the main ways states differ in their education governance 
manifests through the source of selection authority for Boards of Education and Chief 
Education Officials. Having different sources of selection authority can render collaboration 
more difficult because along with these structural divisions come multiple sources of leadership 


and, potentially, conflicting priorities and goals for agencies. 


Table 4.1: Main Models of State Governance (from the Education Commission of the States)?” 


Governance Model 1: Model 2: Model 3 Model 4: 
Model Governor appoints Board is Governor Governor 
the Board of elected appoints Board appoints Board 
Education Board selects Chief is elected Governor 
Board selects Chief Chief appoints Chief 
States Alaska Alabama Arizona Delaware 
Arkansas Colorado California lowa 
Connecticut Kansas Georgia Maine 
Florida Louisiana Idaho New Hampshire 
Hawaii Michigan Indiana New Jersey 
Illinois Nebraska Montana Pennsylvania 
Kentucky Utah North Carolina South Dakota 
Maryland North Dakota Tennessee 
Massachusetts Oklahoma Vermont 
Mississippi Virginia 
Missouri Wyoming 
Oregon 
Rhode Island 
West Virginia 


Table 4.1 Source material: Education Commission of the States, 2013. 


In California, the State Board of Education (SBE)*° plays an integral role in instructional support 
through its constitutional charge to adopt textbooks for grades one through eight and through 
its policy-setting responsibilities.** With its members appointed by the Governor, the SBE is 
responsible for setting policy for academic standards, curriculum, instructional materials, 


39 The Education Commission of the States notes that nine states “function under modified version of the above 
four models.” These states are: Minnesota, Nevada, New Mexico, New York, Ohio, South Carolina, Texas, 


Washington, and Wisconsin. 

40 The California State Board of Education is a durable institution, statutorily established in 1852 and then codified 
in an amendment to the California Constitution in 1884 (California State Board of Education Website: SBE 
Responsibilities, https://www.cde.ca.gov/be/ms/po/sberesponsibilities.asp, accessed December 13, 2017). 

41 California State Board of Education Website: SBE Responsibilities, 


https://www.cde.ca.gov/be/ms/po/sberesponsibilities.asp, accessed December 13, 2017. 
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assessments and accountability, among other responsibilities.** Relatively few other states have 
a State Board of Education that is appointed by the Governor combined with a state 
department of education that is run by an elected official. California shares this model with 
eight other states: Arizona, Georgia, Idaho, Indiana, Montana, North Carolina, North Dakota, 
and Oklahoma (ECS 2013). 


By dividing the source of Board (appointed by the Governor) and Chief (elected by the public) 
selection authority, California’s governance model augments the potential for conflict, which 
many prior reports on California’s education governance have discussed. Further augmenting 
the opportunity for conflict, California’s governors have historically appointed a Secretary of 
Education who reports to the Governor, adding a third state-level education leader. Adding to 
the state-level complexity are two additional state agencies: the Commission on Teacher 
Credentialing and the California Collaborative for Educational Excellence. 


Commission on Teacher Credentialing. Statutorily created as an executive branch agency in 
1970,*3 the California Commission on Teacher Credentialing (CTC) plays a vital role in 
instructional support by setting standards for public educator preparation, by licensing and 
credentialing public educators, and by enforcing its standards, which can include taking 
disciplinary action. Like the California Board of Education, most voting members of the CTC are 
appointed by the Governor.** 


The California Collaborative for Educational Excellence (CCEE). Statutorily created in 2013 as a 


part of the Local Control Funding Formula, the California Collaborative for Educational 
Excellence (CCEE) is a state education agency designed to be part of the California’s larger 
“System of Support” intended to support California districts and schools achieve their Local 
Control and Accountability Goals.*° To date, the CCEE has formed a number of pilot 
partnerships with county offices of education and school districts. They also provide trainings, 


42 “Specifically, ten of the SBE’s eleven members are appointed by the Governor to four-year, staggered terms that 
are subject to confirmation by a two-thirds vote of the Senate within one year of appointment. The eleventh 
member, also appointed by the Governor and subject to Senate confirmation, is a California public high school 
student who serves a one-year term. The student member enjoys full voting rights and all other rights and 
privileges of SBE membership” (California State Board of Education Website: SBE Responsibilities, 


https://www.cde.ca.gov/be/ms/po/sberesponsibilities.asp, accessed December 13, 2017). 


43 The CTC was established in 1970 by the Ryan Act. See https://www.ctc.ca.gov/docs/default- 
source/commission/files/ctc-history.pdf?sfvrsn=96050f5_0, accessed February 27, 2018. 

4 Specifically, the Governor appoints 14 of the CTC’s 15 voting members. The fifteenth voting member is the 
elected State Superintendent of Public Instruction or his/her designee. There are several categories of voting 
members: classroom teachers (6 members), school administrator (1 member), school board member (1 member), 
school counselor (1 member), higher education faculty member (1 member), public (4 members). Higher Education 
entities select the CTC’s four ex-officio, non-voting members (California Commission on Teacher Credentialing 
Website: About the Commission, https://www.ctc.ca.gov/commission/default, accessed December 13, 2017). 

45 California’s “System of Support” also includes the County Offices of Education and the California State 
Department of Education. 
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materials, and technical assistance.*© The Governor’s 2018-2019 budget includes $11.3 million 
of Proposition 98 General Funds for the CCEE “to work with county offices of education to 
provide assistance to school districts, and when necessary, provide direct assistance to school 
districts in specified extraordinary circumstances.”*” The CCEE is governed by a five member 
board whose members include the Superintendent of Public Instruction (or his/her designee), 
the President of the SBE (or his/her designee), a county superintendent appointed by the 
Senate Committee on Rules, a teacher appointed by the Speaker of the Assembly, and a school 
district superintendent appointed by the Governor.*® 


Despite this fragmented state-level governance terrain, interviews of state leaders frequently 
commented on the spirit of collaboration that has emerged during the Brown administration 
and the pivotal role State Board President, Mike Kirst, has played in that process: 


"Under Mike Kirst's leadership, we've worked really hard to have a good 
cooperative relationship with CDE. You may know historically that's not always 
been the case with the state board. We have a really good respectful 
relationship, and | think to Tom Torlakson and Mike Kirst’s credit, they work 
really hard on that. Their teams work very, very hard" (State/Regional Leader 
Interview 038). 


The emergence of California’s “systems of support” approach also received accolades from the 
field: 


“The organization, the cross-agency team [for the statewide systems support] is 
so important ... In my history, that’s the first time I’ve seen all of these other, all 
of these entities coming together in a room and sharing their strategies and 
agreeing upon strategies and sharing their information ... This is critical because 
we’re all getting on the same page and we’re agreeing upon how we’re gonna 
deliver the system in a collaborative manner so that we’re addressing any gaps in 
services” (State/Regional Leader Interview 004). 


Exceptional leadership can help to overcome the challenges that fragmented governance poses. 
California’s governing structure, however, does not inherently lend itself to coordination. How 
might current conditions of collaboration endure? Additional layers of governmental 
complexity offer both promise and additional challenges. 


46 California Collaborative for Educational Excellence Website, http://ccee-ca.org/index.asp, accessed December 

13, 2017. 

47 This $11.3 million reflects $6.5 million “added to $4.8 million in existing funds for 2018-2019.” (Governor’s 

Budget Summary 2018-2019 K-12 Education, 26). 

48 California Collaborative for Educational Excellence Governing Board Bylaws, Article III, Section 1, retrieved from 
ccee-ca.org/documents/Collaborative%20Bylaws%20Adopted%202-25-15.pdf, accessed on December 13, 

2017. 
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4.2 The Sub-State Government Terrain: County Offices of Education and District Networks 
County offices of education and county superintendents feature prominently in California’s 
“System of Support” (Taylor 2017). The Governor’s 2018-2019 budget includes $55.2 million in 
Proposition 98 General Fund for county offices of education to “facilitate the improvement of 
school districts identified as being in need of differentiated assistance.” Recognizing that county 
offices of education vary in their capacity to deliver support, the Governor’s budget also 
includes $4 million in Proposition 98 General Funds “for a competitive grant process to identify 
eight LEA county offices of education, which will provide training, resources, and support for 
other county offices of education” (Governor’s Budget Summary 2018-2019 K-12 Education, 
26). 


In addition to particular county offices of education, the California County Superintendents 
Educational Services Association (CCSESA) provides organizational support across California’s 
varied terrain. County Superintendents offer “direct and regional support to school districts and 
serve as the primary implementation arm of the California Department of Education.”*° Within 
CCSESA, the Curriculum and Instruction Steering Committee (CISC) “identifies statewide 
curriculum and staff development needs, provides a communication and implementation 
network for curriculum and professional development activities, and assists the CDE in adopting 
and implementing instructional materials and developing publications such as curriculum 
frameworks.”°° 


How do administrators, instructional leaders, and teachers navigate this terrain to receive 
instructional support? Networks have emerged as a key component of California’s terrain and 
approach to the puzzles of coordination we list above. As mentioned earlier, we focus on and 
define networks as a “form of collaborative activity for facilitating and operating multi- 
organizational arrangements to solve problems that cannot be solved or solved easily by using 
single organizations” (McGuire 2011, 437-438). We focus primarily on two sorts: information 
diffusion networks and service implementation networks. 


Other resources have catalogued (or have attempted to catalogue) California’s network terrain. 
Rather than provide a comprehensive list, we note here networks that appeared in 
state/regional leaders’ and superintendent interviews. 


Professional Associations. There are many professional networks in California that seek to 
connect teachers and administrators, support the spread of ideas and resources, and provide 
professional development. For teachers, there are state, national, and international 
associations, many of them with a subject-specific focus. Some of the many California teacher 
associations include: 

e Accomplished California Teachers, a National Board Network Affiliate 


49 California County Superintendents Educational Services Association Website: About CCSESA, 


http://ccsesa.org/about/about-ccsesa/, accessed on December 13, 2017. 


5° California County Superintendents Educational Services Association Website: Curriculum and Instruction 
Steering Committee (CISC), http://ccsesa.org/committees/cisc/, accessed on December 13, 2017. 
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e California Association for Bilingual Education 

e California Council for the Social Studies 

e California Mathematics Council 

e California Science Teachers Association, and 

e California Teachers Association’s Instructional Leadership Corp. 


There are also networks for administrators. These include: 
e The Association of California School Administrators, and 
e California County Superintendents Educational Services Association. 


While not all of these professional associations identify themselves explicitly as networks, some 
do; others are described as such by those who participate in them, a few coordinate particular 
networks, collaborations, and communities of practice as a part of their work. 


California Collaborative on District Reform. Housed at American Institutes for Research (AIR), 
the California Collaborative on District Reform convenes several times each year to explore 
problems of district practice. It brings together “researchers, practitioners, policymakers, and 
funders” to engage in “evidence-based dialogue and collective action to improve outcomes for 
all students in California’s urban school systems, with particular attention to equity and access 
for traditionally under-served students in the state.”°? 


The California NGSS K-8 Early Implementation Initiative. The California NGSS K-8 Early 
Implementation Initiative is a partnership between K-12 Alliance at WestEd, the California 


Department of Education, Achieve, eight school districts and two charter management 
organizations: Aspire, Galt Joint Union Elementary School District, High Tech High, Kings Canyon 
Unified School District, Lakeside Union School District, Oakland Unified School District, Palm 
Springs Unified School District, San Diego Unified School District, Tracy Unified School District, 
and Vista Unified School District. Started in 2014, these schools and districts test materials, 
tools, and assessments related to NGSS, and “Learnings from these districts will be shared 
statewide to advance high-quality science teaching and learning.”°? 


California Subject Matter Projects. As we noted earlier, California’s Subject Matter Projects 
represent a long-standing network of support in nine subject-matter disciplines, providing 
professional development to teachers and connecting university faculty, school administrators, 
and teachers in efforts to improve teaching and learning. Today the CSMP still exists and 
operates in roughly 90 regional sites across California despite “changes in funding sources and 
significant declines in overall funding availability” (Bier and Gallagher 2012, 6). 


51 California Collaborative on District Reform Website: About Us, https://cacollaborative.org/about-us, accessed on 
February 28, 2018. 

52 K-12 Alliance Website: NGSS Early Implementers, http://k12alliance.org/ca-ngss.php,accessed on February 28, 
2018. 
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California SUMS Initiative. Scaling Up Multi-Tiered System of Support Statewide is an effort by 
the Orange County Department of Education, in partnership with Butte County Office of 
Education and SWIFT Education Center, to support the statewide implementation of a Multi- 
Tiered System of Support (MTSS). Funded by a grant from the California Department of 
Education, this initiative seeks to implement MTSS statewide through a train-the-trainer model 
using state, regional, county office, and local education agency teams and grants. 


Collaboration Committees and Communities of Practice. Growing out of work from the 
Standards Implementation Steering Committee, which was a joint effort by the State Board of 
Education, the California County Superintendents Educational Services Association, and the 
Department of Education, subject matter collaboration committees and communities of 
practice were formed. The goals of these collaboration committees and communities of 
practice are to support the implementation of standards in the state, to build the capacity of 
county offices to carry out the work, and to support regional networks amongst the county 
offices.°? 


The Consortium for the Implementation of the Common Core State Standards. The Consortium 
for the Implementation of the Common Core State Standards is described by Bill Honig as 


follows: “A potent informal network funded by foundation support, the Consortium for the 
Implementation of the Common Core State Standards, was formed with representatives from 
major educational and government entities, districts, county education offices, teacher groups, 
the research community, higher education, and advocacy groups. It has helped on such key 
issues as implementation planning, coordinating the work of support providers, 
communication, technology, understanding the state mathematics and ELA/ELD frameworks, 
accountability, and new teacher policies. Its first publication, Leadership Planning Guide 
California, was intended to assist districts and schools in addressing the implementation of the 
CCSS. In 2015, the consortium produced user-friendly summaries of the math and ELA/ELD 
frameworks” (Honig 2016). 


The CORE Districts. Founded in 2010, the CORE Districts are located in Fresno, Garden Grove, 
Long Beach, Los Angeles, Oakland, Sacramento, San Francisco, and Santa Ana. This cross-district 
effort has resulted in a comprehensive school improvement and accountability system and 
seeks to “innovate, implement and scale new strategies and tools that eliminate equity and 
achievement gaps and lead to successful outcomes for all students.”°* 


53 In the case of both Science and Mathematics, funding from the S.D. Bechtel, Jr. Foundation allowed for state- 
wide convening of these groups starting in December of 2016 and has led to regional action grants to support this 
work across the state. 

54 CORE Districts Website: Homepage, http://coredistricts.org/, accessed on February 14, 2018. The work of the 
CORE Districts includes data sharing, an improvement community designed to bring improvement science to bear 
on acommon problem of practice experienced in the CORE Districts, and a research partnership with PACE (Policy 
Analysis for California Education). The CORE Districts are funded by the Stuart Foundation, the William and Flora 
Hewlett Foundation, the S.D. Bechtel, Jr. Foundation, and the Bill and Melinda Gates Foundation. 
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Math in Common. Math in Common is a network of ten school districts (Dinuba, Elk Grove, 
Garden Grove, Long Beach, Oakland, Oceanside, Sacramento City, San Francisco, Sanger, and 
Santa Ana). The ten districts are working to support the transition to the Common Core State 
Standards for Mathematics in grades K-8. Their work is supported by California Education 
Partners, an organization that “facilitates collaborations” modeled after teacher-level 
professional learning communities, and WestEd, which is providing research and evaluation.>° 


REACH HIGHER Shasta. REACH HIGHER Shasta is a cradle-to-career initiative that began in 2011. 
The initiative involves the 25 districts in Shasta County, Shasta County Office of Education, two 
charter schools, Shasta-Trinity Regional Occupational Program, First 5 Shasta, local colleges and 
universities, and Shasta County Public Health. It has received support from both the Bill and 
Melinda Gates Foundation and the Carnegie Foundation for the Advancement of Teaching and 
Learning. Particular aspects of this initiative have included a networked improvement 
community to support early childhood reading and an emphasis on counseling in high schools. 


The Regional Assessment Network. The Regional Assessment Network (RAN) is a joint effort 
between the Curriculum and Instruction Steering Committee (CISC) of the California County 
Superintendents Educational Services Association (CCSESA) and the Assessment and 
Accountability Division of the California Department of Education. “The purpose of RAN is to 
develop a knowledge and understanding about California’s assessment and accountability 
systems by facilitating communication between the California Department of Education (CDE) 
and COEs [County Office of Education], which represent the schools, districts, and county 
offices in their region.”°° 


The Rural Professional Learning Network. The Rural Professional Learning Network (RPLN) was 
established in 2015 by Pivot Learning, a non-profit organization in California, with financial 
support from the S.H. Cowell and Hewlett Foundations. RPLN “seeks to identify and alleviate 
local capacity and statewide infrastructure issues that affect rural districts. It does so by 
establishing a network that leverages both in-person meetings and virtual collaboration tools to 
support standards implementation” (Timar and Carter 2017). Some of the first districts to 
participate in the RPLN included Biggs, Durham, Manzanita, and Paradise districts in Butte 
County; Grass Valley, Nevada and Penn Valley districts in Nevada County; Sebastopol district in 
Sonoma County; and the Willows district in Glenn County. 


San Diego Math Network. The San Diego Math Network connects four local school districts 
(Chula Vista Elementary School District, San Diego Unified School District, Sweetwater Union 
High School District, and Vista Unified School District) and a few additional partner schools with 
UC San Diego in order to support K-12 leaders and mathematics teachers. The network has a 
strong focus on elementary and middle school math instruction and the transition for students 


55 This network is funded by the S.D. Bechtel, Jr. Foundation. 
5° California Department of Education Website: Regional Assessment Network: 


https://www.cde.ca.gov/Ta/Tg/ai/caregionalranmap.asp, accessed on February 14, 2018. 
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between elementary and middle school mathematics, and they organize “public ‘Learning 
Events,’ led by San Diego educators to explore issues of critical regional importance in 
mathematics.”°” 


How do practitioners navigate the governmental terrain? In educational contexts like 
California’s, networks can offer a range of contributions, including information exchanges and 
flexible support (Goldsmith and Eggers 2004). State/regional leaders and superintendents 
focused primarily on the assets networks offered. Some respondents flagged how district 
networks stepped into spaces where gaps in state services and support appeared: 


“The existence of these CORE districts ... is really testimony to the fact that the 
districts have gotten together and said, ‘We’re gonna solve this problem 
ourselves.’ And in the leading cases, places like Long Beach, Fresno, Sacramento, 
to an extent San Francisco and Oakland, have gotten into really interesting deep 
sharing, over techniques and practices, they trade second and third level 
administrators back and forth a lot, and so they’ve got this whole environment 
that’s working, will it continue?” (State/Regional Leader Interview 028) 


The CORE districts provide an example of formal informational network, sharing techniques, 
practices, and even administrators. The CORE districts’ data collaborative, moreover, has 
provided opportunities to develop and maintain internal data systems intended to support their 
efforts of continuous improvement, since current education data architectures may lack 
coordination across state organizations and levels of the education system (Phillips, Reber, and 
Rothstein 2018). Regional collaborations, piloted by the CORE districts, build both data systems 
and aspects of professional development focused on data use spanning counties (Phillips, 
Reber, and Rothstein 2018; Hough, Byun, and Mulfinger 2018). Considering capacity building 
for data use alongside the creation of data systems is important since “data do not objectively 
guide decisions on their own” (Spillane 2012, 114) and many factors influence that use 
including interpretive processes, social and organizational conditions, and power relations 
(Coburn and Turner 2011). 


Superintendents noted other formal networks as well, and how these networks provided them 
with helpful information and approaches to addressing the problems their districts faced: 


“{Our network] helps us assess how we’re assessing curriculum. It helps open up 
the thinking around how we’re assessing and looking at evidence of learning of 
all different kinds of students. That’s what outside resources and institutions 
help us do. And to look at and try new frameworks and conceptual models when 
things aren’t moving forward the way we want. Ways we can take a conceptual 


57 San Diego Math Network Website: About Us: http://sdmn.ucsd.edu/about-us/,/, http://sdmn.ucsd.edu/about- 
us/, accessed on February 14, 2018. The network was initially created with funding from the Bill and Melinda Gates 
Foundation in 2015. 
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framework or a model and turn it into action learning or action research. It’s 
always beneficial for teams across the district” (Superintendent Interview 160). 


Superintendents also noted that they frequently consult informal networks: other 
superintendents whom they contact for advice. Returning to the same material we presented 
earlier in Figure 3.6, recall that the superintendents we interviewed were more likely to receive 
information on professional development from other superintendents than from any other 
source. 


Yet, networks are not unalloyed goods (Goldsmith and Eggers 2004). While state/local leaders 
focused on networks’ assets, they flagged concerns about network overload, isolation, and 
ambiguity. With their horizontal (rather than vertical) structure, networks can raise issues of 
responsibility (who decides what to do) and who to call for help. And, even in an era of 
“systems of support” knowing who to call and what to do is still evolving: 


“The tension has always been present in regional experiences and structures. 
The county superintendents are saying, 'Look, you work for me. We want you to 
do this.' The Department is saying, 'We fund you. We want you to do this." | think 
there's been some evolution in that thinking at this particular juncture of 
California education, for agencies to think about regional work in a more 
coherent way...The Department has been very good about heading in that 
direction. At this time, is a new agency for support, separate from the 
Department, and there are the variations of need given the local contexts. How 
are those pieces really going to fit together in a coherent way that allows 
districts to experience valuable technical assistance? How does a district team 
know who to call and what to do for support, not sure there is clarity on the 
processes now?” (State/Regional Leader Interview 006) 


As this state leader noted, the Systems of Support model and the creation of the CCEE reflected 
an effort toward greater coherence in support. Yet, none of the 91 superintendents interviewed 
for this study raised and discussed the CCEE as an important source of instructional support.°® 
Only a few California state/regional interview respondents raised the CCEE in open-ended 
discussion of California’s structures for instructional support, and most did so to raise concerns 
about the CCEE’s capacity to reach the scope and scale of California’s instructional support 
needs. 


“CCEE is currently exploring different strategies including professional learning 
networks to increase school and district success. Participants indicate that this 
smaller group work is helpful but taking such a strategy to scale across the state 
would be difficult if not impossible to fund given our size” (State/Regional Leader 
Interview 006). 


58 However, the Superintendent Interview Protocol did not specifically ask superintendents about the CCEE. 
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Moreover, knowing “who to call and what to do” is still a work in progress, as is relying on 
county offices to be a main provider of direct technical assistance. Along with support for 
appropriations to county offices to provide instructional support come calls for accountability: 


“| would prefer to see some strengthening around the accountability for county 
offices in terms of how we use those funds and support school districts. | think 
that we—all of us need to be held accountable, right?” (State/Regional Leader 
Interview 004) 


And, along with investment in county offices as a chief vehicle for instructional support 
come questions of what to do about the variability in county office quality and district 
isolation from supportive networks. 


“| really feel that we are left to figure it out on our own. Education now is solely 
focused on mandates, and it [the state] really isn’t a source of support. The 
county doesn’t have resources, or doesn’t use resources, for staff development. 
We have a lot of different things happening across the state in isolation, but it is 
sort of hit or miss. | belong to some great superintendent groups, so that’s 
where | get the vast majority of my ideas or from ACSA" (Superintendent 
Interview 024). 


“We have a lot of different things happening across the state in isolation, but it is sort of hit or 
miss” goes to the heart of the technical, organizational, and political puzzles that we raised. A 
lot is happening across the state: the sector of education service providers has ballooned over 
the past decade. But, to what extent is this expanding sector “hit or miss” and who, in 
particular, gets missed? 


4.3 The Terrain of Governmental and Non-Governmental Providers 

The elements that would need to be coordinated to build a coherent system are those that 
have a strong influence on the key components of guidance for instruction, namely academic 
standards, assessments, instructional materials, and teacher education/professional 
development. Again, we focus primarily here on materials and professional development, and 
the challenge to provide centralized support—to manage the “noise” problem—to enable 
robust decentralized decision-making. 


"There’s a ton of ... noise in California. When you work in the district, there’s a 
lot of different companies and agencies from California, from outside of 
California, non-profits and for-profits, all of them bombarding you with ideas of 
here’s what you need to do to improve student achievement. So, it’s sometimes 
difficult for district to sort those things out and not succumb to the temptation 
of name recognition ...” (State/Regional Leader Interview 045). 


Materials. The State Board of Education approves a list of instructional materials that meet its 
standards of quality. California’s list is not as curated as some other states, like Louisiana’s. 
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Some superintendents noted how curating the vast terrain of materials could be part of a 
deliberate process “to ensure that the practitioners are getting the opportunity to preview and 
test drive materials that are out there to determine if they’re effective” (Superintendent 
Interview 159). Others spoke of deliberate collaborations with other districts to support the 
curating process. 


“We have a curriculum department and they’re connected with other curriculum 
departments across the state and beyond. There’s a fairly robust conversation 
among curriculum and instructional folks about how people are implementing 
the new state standards” (Superintendent Interview 132). 


Many other superintendents noted the amount of time they spend curating the new terrain of 
materials and their frustrations with the quality that they encounter.°? 


“There is so much out there that there needs to be more vetting in the sense of 
what is state approved” (Superintendent Interview 025). 


“There’s a lot out there but it’s not all good. We’re looking at links into lesson 
plans ... that take a teacher directly to the resource that they need that’s been 
vetted and approved for use” (Superintendent Interview 187). 


“The state also has a list of approved materials, which is useless because some of 
the material is quite lacking in terms of quality... we are dependent on our own 
internal analysis” (Superintendent Interview 016). 


Recall that one puzzle we raised is how to coordinate the actions and products of many 
independent private sector curriculum designers, publishers, and materials providers so that 
they deal with the same knowledge and skills? Districts reported frustrations not only with the 
state-approved list, but also with the challenges they face assessing the quality of information 
they receive from materials’ providers and publishers. 


“They also get a fair amount of information from the publishers of the 
curriculum. That’s where you get the real disconnect between someone who 
says their materials are aligned but now we need to talk about what your 
definition of alignment is. They’re attempting to sell a product, schools are trying 
to fill a need, there’s not an unlimited amount of time and resources, and so 
everyone tries to get the best info they can” (Superintendent Interview 132). 


5° These frustrations are consistent with earlier reports of materials-related challenges to Common Core 
implementation in California (McLaughlin et al. 2014). They also parallel findings by Gao et al. in a survey 
conducted in 2017, examining districts’ implementation of the Next Generation Science Standards (NGSS) 
framework (Gao et al. 2018, 12). 
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The burdens of curating materials can be especially acute for small and/or under-resourced 
schools. 


“A 3,000-student school district does not have the capacity to judge 25 approved 
math materials. It's just not possible. You have to have some kind of sieve, and 
then you have to figure out a way, when they select materials, to actually give 
them some level of support. There's no infrastructure in California for them to 
do that” (State/Regional Leader Interview 018). 


Professional Learning Opportunities 

Another puzzle we raised is how to coordinate the actions and products of many independent 
private and public-sector teacher education and professional development agencies so that 
they instruct teachers in the same knowledge and skills. Research suggests that materials alone 
are unlikely to significantly improve instruction and that professional development alone is also 
unlikely to significantly improve instruction (Hill and Moffitt 2017). The two together, 
coordinated, is more likely to improve instruction (Saxe et al. 2001; Roschelle et al. 2010; Tong 
2014; Lara-Alecio 2012). Superintendents noted the struggle they faced obtaining both quality 
instructional materials and professional learning opportunities to use those materials. 


“The new standards are wonderful, especially math standards, but folks will 
need a lot of assistance to make that shift [...] Often times, when new standards 
come in, the next step is how we are going to provide instructional materials, 
and this requires a fair amount of professional development. The professional 
development needs to be ongoing, and we need to keep returning to it so we 
can implement standards” (Superintendent Interview 016). 


This superintendent went on to note the burden that this can place on under-resourced schools 
and districts. “If you make a [materials] selection that would require a level of professional 
development that you do not have the resources for and your staff is not prepared, you are 
going to run into issues there” (Superintendent Interview 016). The issue, this superintendent 
raised, is not only curating materials for quality but receiving information and support on the 
amount of professional learning those materials will require to be useful in practice. 


When it comes to support for professional learning opportunities, California, like many other 


states, relies heavily on its non-governmental sector to provide services. Figure 4.1 reveals the 
growth in non-profit education service organizations in California over the past 15 years. 
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Figure 4.1: Number of Non-Profit Education Service Organizations, 
CA 
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Figure 4.1 Source: National Center for Charitable Statistics Core Trend Data, 2000 — 2015. To create a list of organizations 
providing services in education we focused on the data by the NCCS primary purpose classification: 1) research institutes and 
public policy analysis, 2) special education, 3) libraries, 4) student services, 5) educational services, and 6) remedial reading and 
encouragement. We then checked each organization to insure it focused on public education in the state of California. Please 
see the Appendix for source information. 


Figure 4.2 suggests a similar trend throughout the United States over the past fifteen years. 


Figure 4.2: Number of Non-Profit Education Service Organizations, U.S. 
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Figure 4.2 Source: National Center for Charitable Statistics Core Trend Data, 2000 — 2015. To create a list of organizations 
providing services in education, we focused on the data by the NCCS primary purpose classification: 1) research institutes and 
public policy analysis, 2) special education, 3) libraries, 4) student services, 5) educational services, and 6) remedial reading and 
encouragement. Please see The Appendix for source information. 
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How is the expanding California terrain of non-profit education service providers distributed? 
Our interviews raised concerns that relying on non-governmental providers could advantage 
urban areas over rural areas: 


“| think one thing that we always forget that with nonprofit and for-profit 


providers, their services tend to be more aligned in the urban areas because 
that’s where the most fitness is” (State/Regional Leader Interview 015). 


Figure 4.3 Distribution of CA Non-Profit Education Service Organizations, 2015 


Figure 4.3 Source: National Center for Charitable Statistics Core Trend Data, 2015. NCCS organization data was geocoded by zip 
code in ArcGIS. Map produced using ArcGIS. Please see The Appendix for source information. 


Potential disparities in the distribution of non-governmental service providers offer grounds for 
investing in the county-system of instructional support. 
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“We know that there are 58 counties, 35 are rural, so if you want high quality 
support to be available in say Tehama County, Siskiyou County, Del Norte, 
Humboldt, you have to invest in a county office structure. That is not to say that 
non-profits and for-profits don’t have a role in any of this work. It’s one of those 
where as a state we have a responsibility to ensure that every district in our 
state has access to high quality support for their students and the free market 
right now will not provide that because we know that’s not the way business 
works. | think that that’s one piece that has to be a high priority for the new SPI 
and | think the path is in place forward on that, so that’s a good thing” 
(State/Regional Leader Interview 015). 


This concern, that non-governmental providers tend to be more established in urban areas than 
in rural areas is borne out by the map of non-profit educational service providers in Figure 4.3. 


Yet, potential inequities in the distribution of educational service providers appear on a second 
dimension as well. In addition to the absence of coverage in some areas, the data also suggest 
insufficient coverage, given student enrollment size and needs. Figure 4.3 reveals the lack of 
coverage in rural areas. Table 4.2 reveals that, in addition to a lack of access in rural areas, 
some urban counties may be under-served. 


Further, Table 4.2 reveals discrepancies in organizational capacity in counties with large 
populations of students who qualify for free or reduced price lunch and identify as English 
Language Learners. Outside of urban counties, low-income counties with large populations of 
ELL students have limited access to education service organizations. 


At the organizational level, similar discrepancies in coverage emerge: in our review of mission 
statements for education service organizations, only 1 percent of organizations explicitly 
mentioned English Language Learners, 9 percent mentioned low-income students, and 4 
percent of organizations mentioned at-risk student populations in their mission statement.°° 
While many organizations likely target these populations, this descriptive data presents a lack 
of targeted resources to high-need student populations. 


5° Our estimate is conservative. We included only organizations that explicitly mentioned targeting these 
populations in their mission statements. 
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Table 4.2: County-Level Counts of Non-Profit Education Service Organizations (NCCS, 2015), by 
Student Population and Demographic Characteristics (SAIPE and CDE, 2015) 


No. of Pct of 
Orgs, per Pop. 5-17 Pct of 
100,000 in Students, Median HH 
County Name students No. of Orgs Total Org. Revenue Poverty ELL Income 
Alameda County 8.0 18 $12,058,422 13.4% 21.9% $81,462 
Butte County 3.2 1 $880,129 22.6% 8.8% $45,369 
Calaveras County 17.4 1 $3,500 18.6% 2.4% $52,471 
Contra Costa County 2.3 4 $947,982 11.6% 17.7% $83,036 
El Dorado County 3.7 i $99,813 10.4% 7.4% $75,575 
Fresno County 1.0 2 $431,122 33.6% 22.6% $46,608 
Humboldt County 5.5 1 $3,512 22.8% 7.3% $40,739 
Kern County 1.1 2 $7,316,113 29.2% 22.0% $51,150 
Lake County 10.9 1 $10,791 30.0% 12.3% $37,993 
Los Angeles County 3.2 50 $60,239,091 22.8% 22.7% $59,045 
Madera County 3.2 1 $30,437 31.1% 25.7% $46,593 
Marin County 9.0 3 $646,730 7.7% 15.0% $99,868 
Mendocino County 15.4 2 $78,222 25.2% 21.2% $43,237 
Monterey County 5.3 4 $815,778 21.5% 41.2% $60,047 
Napa County 14.3 3 $3,163,595 11.0% 23.1% $72,683 
Nevada County 8.1 1 $304,861 14.8% 5.3% $57,627 
Orange County 1.8 9 $12,220,211 16.2% 26.0% $78,002 
Placer County 2.8 2 $620,892 8.1% 8.2% $76,203 
Riverside County 0.5 2 $216,846 21.5% 20.9% $57,895 
Sacramento County 2.1 5 $4,159,706 21.8% 18.1% $58,735 
San Bernardino County 1.7 7 $6,681,206 26.4% 19.2% $53,526 
San Diego County 3.0 15 $15,922,185 17.2% 22.4% $67,053 
San Francisco County 23.7 14 $24,514,704 14.5% 27.8% $90,527 
San Luis Obispo County 2.9 1 $113,432 13.8% 15.6% $61,761 
San Mateo County 8.4 8 $11,336,786 9.1% 24.4% $101,133 
Santa Barbara County 2.9 2 $219,505 18.8% 35.0% $63,049 
Santa Clara County 5.4 15 $16,231,190 8.5% 24.1% $102,191 
Shasta County 3.8 1 $23,169 24.0% 3.3% $45,943 
Solano County 1.6 1 $25,973 14.9% 13.8% $67,202 
Sonoma County 2.8 2 $1,039,893 12.8% 23.2% $66,463 
Stanislaus County 0.9 Z $157,906 25.3% 25.0% $51,949 
Tehama County 9.5 1 $50,137 29.5% 16.7% $40,292 
Tulare County 1.0 1 $40,299 34.6% 28.2% $42,637 
Ventura County 2.1 3 $2,046,385 13.5% 23.8% $79,285 
Yolo County 3.4 1 $587,122 15.0% 21.6% $58,766 


Table 4.2 Notes: This data is derived from several sources: first, we merged the geocoded organization for fiscal year 2015 by zip code to the 
county level boundaries from the state of California. Next, we merged the organizational data with aggregated county estimates of English 
Language Learners, from the California Department of Education for the 2015 school year. Poverty estimates are based on the percent of the 
population aged 5-17 in poverty, from the Small Area Income and Poverty Estimates (SAIPE) in 2015. The table excludes counties that do not 
have service organizations according to our analysis, e.g. any county that did not have organizations geocoded within the county boundary 
(there are 35 counties with organizations out of 58). 
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Figure 4.4: Distribution of CA Private Foundations and Grant Recipients, 2014 
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Figure 4.4 Source: Open Center for Nonprofit Research, 2014. Organization data was geocoded by zip code in ArcGIS. Map 
produced using ArcGIS. There are 778 grants included in the sample, and 458 foundations. The foundation data was merged 
with aggregated county estimates of English Language Learners, from the California Department of Education for the 2015 
school year. Poverty estimates are based on the percent of the population aged 5-17 in poverty, from the Small Area Income 
and Poverty Estimates (SAIPE) in 2015. The table excludes counties that do not have foundations or grant recipients service 
organizations according to our analysis, e.g. any county that did not have organizations geocoded within the county boundary 
(there are 41 counties with organizations out of 58). Please see the Appendix for source information. 


Discrepancies in philanthropic support compound issues of political and organizational capacity. 
Figure 4.4 displays the geographic distribution of private foundations and grant recipients® for 
2014. 


51 This data includes all private foundations that filed the IRS Form 99PF electronically in 2014, which is 
approximately 60% of all organizations. The map shows foundations that gave to grant recipients focused 
improving education in public schools and districts. Although private foundations that filed a form 990PF by mail 
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Table 4.3: County-Level Counts of Private Foundations and Grant Recipients (NCCS, 2014), by Student 
Population and Demographic Characteristics (SAIPE and CDE, 2015) 


are excluded from the analysis, we do not believe this skews our findings because we do not expect there to be 
regional variation in the types of organizations that file taxes online. For a more thorough review of the data, 
please refer to The Appendix. 
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No. of Grantees, per Pct of Pop 5- Pct of Students, Median HH 

County Name PF, per 100k Students No. of PF Grantees 100k Students Total Grant Amount 17 in Poverty ELL Income 
Alameda County 11.1 25.0 46.2 104.0 $2,126,780 13.4% 21.9% $81,462 
Amador County 24.8 1.0 24.8 1.0 $4,000 17.2% 2.4% $55,879 
Butte County 6.4 2.0 3.2 1.0 $1,000 22.6% 8.8% $45,369 
Contra Costa County 10.3 18.0 19.4 34.0 $420,625 11.6% 17.7% $83,036 
El Dorado County 0.0 0.0 3.7 1.0 $1,000 10.4% 74% $75,575 
Fresno County 5.0 10.0 6.0 12.0 $245,500 33.6% 22.6% $46,608 
Glenn County 0.0 0.0 88.2 5.0 $12,950 23.4% 23.6% $43,584 
Humboldt County 0.0 0.0 5.5 1.0 $10,000 22.8% 7.3% $40,739 
Imperial County 5.4 2.0 37.6 14.0 $20,322 32.1% 43.3% $39,925 
Inyo County 38.5 2.0 77.0 4.0 $40,005 19.2% 14.2% $51,697 
Kern County 44 8.0 6.7 12.0 $354,669 29.2% 22.0% $51,150 
Kings County 0.0 0.0 7.0 2.0 $1,500 28.1% 21.9% $46,440 
Lake County 0.0 0.0 10.9 1.0 $1,250 30.0% 12.3% $37,993 
Los Angeles County 16.0 247.0 26.4 407.0 $14,950,892 22.8% 22.7% $59,045 
Madera County 0.0 0.0 3.2 1.0 $1,000 31.1% 25.7% $46,593 
Marin County 60.2 20.0 108.4 36.0 $907,170 7.7% 15.0% $99,868 
Mariposa County 0.0 0.0 53.5 1.0 $1,000 23.2% 3.2% $44,595 
Mendocino County 15.4 2.0 92.2 12.0 $68,027 25.2% 21.2% $43,237 
Merced County 1.8 1.0 10.5 6.0 $4,000 36.3% 28.3% $42,879 
Modoc County 0.0 0.0 69.6 1.0 $832 26.8% 20.2% $38,402 
Monterey County 14.5 11.0 31.6 24.0 $280,833 21.5% 41.2% $60,047 
Napa County 33.3 7.0 33.3 7.0 $65,000 11.0% 23.1% $72,683 


Nevada County 0.0 0.0 16.3 2.0 $9,100 14.8% 5.3% $57,627 
Orange County 12.7 63.0 18.1 90.0 $1,515,688 16.2% 26.0% $78,002 
Placer County 9.9 7.0 71 5.0 $2,456 8.1% 8.2% $76,203 
Riverside County 1.2 5.0 2.8 12.0 $129,856 21.5% 20.9% $57,895 
Sacramento County 3.7 9.0 6.2 15.0 $154,283 21.8% 18.1% $58,735 
San Bernardino County 1.2 5.0 2.4 10.0 $55,333 26.4% 19.2% $53,526 
San Diego County 8.3 42.0 19.0 96.0 $3,364,400 17.2% 22.4% $67,053 
San Francisco County 108.3 64.0 186.1 110.0 $3,099,625 14.5% 27.8% $90,527 
San Joaquin County 49 7.0 76 11.0 $66,517 23.1% 23.1% $53,341 
San Luis Obispo County 20.1 7.0 48.9 17.0 $124,420 13.8% 15.6% $61,761 
San Mateo County 46.2 44.0 85.1 81.0 $1,144,153 9.1% 24.4% $101,133 
Santa Barbara County 36.4 25.0 64.1 44.0 $485,861 18.8% 35.0% $63,049 
Santa Clara County 23.1 64.0 48.8 135.0 $3,650,194 8.5% 24.1% $102,191 
Santa Cruz County 12.3 5.0 34.5 14.0 $303,736 16.5% 29.4% $64,841 
Shasta County 3.8 1.0 11.3 3.0 $1,500 24.3% 3.3% $45,943 
Sierra County 268.8 1.0 268.8 1.0 $500 14.5% 6.2% $47,535 
Siskiyou County 0.0 0.0 17.5 1.0 $2,000 29.2% 3.4% $37,447 
Solano County 1.6 1.0 9.4 6.0 $22,500 14.9% 13.8% $67,202 
Sonoma County 9.8 7.0 32.3 23.0 $327,771 12.8% 23.2% $66,463 
Stanislaus County 3.7 4.0 5.6 6.0 $36,554 25.3% 25.0% $51,949 
Tehama County 0.0 0.0 9.5 1.0 $10,000 29.5% 16.7% $40,292 
Tulare County 1.0 1.0 49 5.0 $115,000 34.6% 28.2% $42,637 
Ventura County 6.3 9.0 18.3 26.0 $876,862 13.5% 23.8% $79,285 
Yolo County 6.8 2.0 10.2 3.0 $8,500 15.0% 21.6% $58,766 


Table 4.3 Notes: This data is derived from several sources: first, we merged the geocoded organization from the Open Center for Nonprofit 
Research, fiscal year 2014 by zip code to the county-level boundaries from the state of California. Next, we merged the organizational data with 
aggregated county estimates of English Language Learners, from the California Department of Education for the 2015 school year. Poverty 
estimates are based on the percent of the population aged 5-17 in poverty, from the Small Area Income and Poverty Estimates (SAIPE) in 2015. 
The table excludes counties that do not have service organizations according to our analysis, e.g. any county that did not have organizations 
geocoded within the county boundary (there are 35 counties with organizations out of 58). 


We focus on foundations that directly fund public schools and districts, and plot the location of 
the grant recipients to illustrate the dispersion of private funding across the state. Foundations 
are primarily located in urban areas, and importantly, their philanthropic investment focuses on 
urban areas. Table 4.3 reveals patterns of foundation investment across counties. Philanthropic 
investment is focused in counties with relatively low rates of child poverty and relatively high 
household income. There is wide variation in philanthropic investment, as well: in Los Angeles 
county, grant funding is about $344 million. In contrast, rural counties like Shasta and Sierra 
receive $1,331 and $268, respectively. In addition, urban counties have more foundations and 
grant-recipients, contributing to the imbalance in organizational and political capacity. 
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Networks offer one way of mitigating the geographic isolation that can arise from place-based 
non-profit educational service provision. Regional networks, in particular, built on the county 
office structure offer one path: 


“| liked the idea of the current idea of how networks are organized under the like 
professional area or a focus area, but | also believe there’s a need to have 
regional networks ...” (State/Regional Leader Interview 004). 


Specialization is important, this state leader continued, otherwise “you just spread yourself too 
thin” (State/Regional Leader Interview 004). And regionalization can help expand access to 
information resources beyond traditional county lines: 


“If I'm needing somebody who’s a specialist in ELD, why can't | call [another 
county] and say, ‘We’d like to contract with you for coming in and providing this 
particular set of, you know, institutes or conversely, working with this particular 
district?’ You know, we need a coach” (State/Regional Leader Interview 004). 


Counties, however, vary in their capacity to deliver instructional support. Again, capacity 
emerges relative to the task at hand: 


“Some [county offices] are just so small ... They don’t have very many people. 
The people that are there, who may be very good, just have quite a lot of 
responsibilities, very similar to small school districts” (State/Regional Leader 
Interview 006). 


Districts also require capacity to be able to engage with network opportunities. In addition to 
the absence of potential connections or few connections relative to need, districts and counties 
also raised the issue of network or information overload: 


“Districts have to be selective about what they participate in. And so it becomes 
a challenge for our smalls. So if you are the superintendent, principal you can’t 
possibly be attending all these different [opportunities]. Nor do you have a large 
enough staff to delegate and send folks. So how can we consolidate some of that 
information and get it out so that it gets to our small schools?” (State/Regional 
Leader Interview 029) 


Returning to our core organizational puzzles confronting the construction of coherent 
structures of instructional support, what are the current conditions? 


e We find exceptional leadership that has navigated the structural divides inherent in the 


design of state-level agencies, along with concerns about the durability of the gains in 
cross-agency collaboration. 
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e We find a vast terrain, with “a lot of different things happening.” Some of those 
happenings embody the principles and practices of building coherent structures of 
instructional support. Others operate “in isolation” or are “overwhelmed” by having to 
curate the expanded space themselves.*°* 


4.4 Opportunities to Learn: Keeping an Eye on Content 

We focus on networks as a “form of collaborative activity for facilitating and operating multi- 
organizational arrangements to solve problems that cannot be solved or solved easily by using 
single organizations” (McGuire 2011, 437-438). Networks are, by design, content-free. The 
content they convey depends on who they connect, what their connected participants share, 
and how they share it. While networks allow for their organizational and individual participants 
to engage in processes of continuous learning, that learning also depends on the quality of the 
content or information that the networks transmit.°? 


California’s embrace of decentralization expressed through LCFF/LCAP is consistent with a 
networked approach. Both place faith in grassroots problem-solving. This approach has a long 
history in American education policy (Elmore 1979) and in American governance (De 
Tocqueville 2002). California’s move toward decentralization coincides with its call for more 
rigorous instruction, which implicates content. Yet, collectively, the educational terrain remains 
far from the content envisioned in its standards. In the words of one network leader: 


“When we think about the system, and all the things that go into the system to 
support quality education, the standards are just one part of that ... you first 
have to have a vision of what excellent instruction means. It’s just amazing how 
far away people are from that” (State/Regional Leader Interview 002). 


Not only is the content of ambitious instruction and learning hard, so is the process of local 
control and engagement (Humphrey et al. 2017, 24). 


“| know how difficult strategic planning can be for any organization or any group 
of people to come together and identify their goals, implement corresponding 
actions and services, agree on what their outcomes are, and agree what the best 
indicators are for those outcomes. That's a hard project for any organization of 
any size. California said, ‘Alright, let's do local control funding formula and we're 
just going to ask a thousand districts in the state to do this kind of mind- 
boggling, complex strategic plan and we're going to ask them to involve 
stakeholders in ways that many have little or no experience doing and, by the 


52 Some states curate their materials more rigorously than California. Louisiana, for instance, provides a tiered 
system of reviews of textbooks and instructional materials. This may help account, in part, for the greater 
coherence identified in teachers’ instructional practice in Louisiana, according to a recent Rand study (Kaufman et 
al. 2016). Responsibility for following a different materials review process resides with the State Board of 
Education. 

53 In addition to content being important, there are still knowledge and skills to be gained in the state about what 
is meant by continuous improvement (Nayfack et al. 2017; Humphrey et al. 2017). 
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way, we're going to be designing the accountability system as we go along, and 
we're going to research the hell out of you while you're doing it.’ It's no surprise 
that there was kind of mixed results around that” (State/Regional Leader 
Interview 017). 


Both content and process are key elements in structures for instructional support. How might 
California deliberately stitch together its efforts at promoting rigorous content with its efforts 
at continuous improvement? Investing in existing subject-specific network strategies like 
Communities of Practice and the Subject Matter Projects that connect subject-matter content 
with improvement processes offer potential foundations on which to build. They have a 
network infrastructure. They have a strong content-base. 


The Subject Matter Projects are, however, “much less substantially funded than they once 
were” (State/Regional Leader Interview 037) and increasingly rely upon a pay-for-service 
model. Bier and Gallagher (2012) point out that while this cost recovery model “may make the 
CSMP more responsive to its partners, it also raises challenges of maintaining high-quality and 
relatively long duration programs that meet all educators’ needs” (6), especially when “districts 
and schools are more interested in hiring the CSMP for short-term assistance in implementing 
curriculum” (6). Reinvesting in Subject Matter Projects could afford the state an opportunity to 
build on an existing foundation that combines content and process and build on ties with higher 
education. 


Whether it builds on the SMP structure or on another structure, such as the Collaboration 
Committees or Communities of Practice, the state should bear in mind the challenges that arise 
with networked approaches, especially the challenges they pose to covering and meeting 
California’s full and varied socio-economic, geographic, and ethno-linguistic terrain of 
educational needs. 


4.5 Opportunities to Learn: Research and Strategy to Support Process and Content 
Our interviews with state-level and regional-level leaders also revealed unmet demand for 
applied research to support instructional practice. 


“So, one of the things that we’re looking to the CCEE to do and the CDE is to 
really vet the research-based strategies programs and practices ... how do we 
know that XYZ is a quality evidence-based program that makes a difference with 
that particular population? ... That research and that vetting is absolutely 
necessary and it’s supportive to us, the vetting of the interventions” 
(State/Regional Leader Interview 004). 


This vetting or applied research support can complement the state’s shift toward 


decentralization: for the central state to support local practice by managing the “noise” 
the frontlines face from service and materials providers. 
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“While we're giving local flexibility, there's still room for good examples to be 
called out which should come from the data” (State/Regional Leader Interview 
012). 


Recall the state leader’s observation above that: 


“The CDE should grow to be a support organization, but ... it’s always gonna be 
indirect support by facilitating the subject matter projects, by facilitating the 
county office of education, by facilitating nonprofit providers to play that role of 
increasing the quality of support out there” (State/Regional Leader Interview 
015). 


As a support organization, how might the CDE advance the content and process of instructional 
improvement? Returning to our opening concepts, how might it do so in complementary rather 
than zero-sum fashion? Tennessee’s Division of Research and Strategy offers a model of how a 
state agency can structure itself as a learning organization, and forge partnerships with other 
organizations to create products that inform both policy and practice.© The Tennessee 
approach is not just a state longitudinal data system. Consistent with ample evidence that data 
alone—at any level of governance—are insufficient to improve practice (Spillane 2012; Coburn 
and Turner 2011), Tennessee’s model creates a structure that explicitly connects flexible and 
timely data to the state’s policy agenda and strategic priorities (Conoway et al. 2015). 


Some key elements of the Tennessee Division of Research and Strategy design:® 


It is financed through both state and federal funds, but is not responsible for 
managing federal reporting requirements. This allows the Division both the opportunity 
to set the terms of its research and strategy agenda and to have the time to do their 
work. Scholarship frequently attends to the ways in which federal funding can set the 
priorities of subnational organizations. In addition to federal goal setting, however, 
administering federal grants and fulfilling the reporting requirements that come with 
federal grants can be enormously time consuming and can crowd out other work. 


It is housed in the state agency as a distinct division. It is not combined with 
accountability, which buffers the agency from the political firestorms that can erupt 
around matters of accountability. This also allows the division to attend agency-wide 
learning and policy learning. 


It employs in-house experts in research and strategy. Those in-house experts are then 
well-equipped to construct and implement work agreements with outside units and 
contractors who perform some of the research. The division, thus, avoids the “hollow 
state” phenomenon. 


54 On the CDE becoming a “data hub” to support Common Core implementation, see Warren and Murphy (2014). 
55 For more on creating state data systems that are actually useful to policy and practice, see Conoway et al. 2015. 
56 State/Regional Leader Interview 033. 
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e It systematically connects with all parts of the department to understand the issues 
facing the other parts of the agency and to learn how the Division of Research and 
Strategy’s work can be helpful. 

“We intentionally restructured ourselves so we had within our team little 
mini-teams with different focus areas [such as special education, teacher 
evaluation, or teachers and leaders]. Each team now has a manager and 1 
to 4 people. The manager makes sure that they are deeply tapped into all 
the decisions with the division with which the person is working. This 
allows [the Division of Research and Strategy] to be embedded in all of 
the work throughout the department” (State/Regional Leader Interview 
033). 


So, how does the agency do its work? Tennessee’s Division of Research and Strategy structures 

itself and it operates as a learning organization; and it enables the state and its policies to do 

the same. 

e It systematically learns about the learning-needs of different parts of the department. 

“In the month of July, we go around and have meetings with every one of our 
leadership teams in different divisions. They have conversations about ‘What are 
the things your division is trying to do over next 6 months? What are the big 
decisions that are coming up?’” (State/Regional Leader Interview 033) 


e It formulates research questions and products so they meet the needs of the state and 
the different divisions within the state agency. The Division of Research and Strategy 
chooses questions and produces research that “answer useful and interesting questions 
for people, that give them real information” that they can use in their division in the 
near-term. It is designed as a learning organization that supports other units’ learning. 
The Division explicitly considers potential implications from its research, and it lays the 
groundwork other parts of the agency to take subsequent action. Doing this on the 
front-end makes a substantive difference in the Department’s ability to ultimately take 
action. 


e It recognizes that structure matters. 
“The structure that TN put in place allowed a single team to have access 
to all the datasets they needed, and to have the ability and know-how to 
take data sets, but put them together in new ways, give answers to new 
questions, and go talk to people until we had answers and could accrue 
knowledge” (State/Regional Leader Interview 033). 


e Itlearns from experience. The Division of Research and Strategy structures itself as a 
learning organization, to learn from its experiences. 
“TAt its annual retreat, the Division] thinks about what makes for a high- 
quality project. We spend time collecting things outside of TN that we 
wish we had done. And we look back at our projects from the past year: 
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what worked, what did not. This is partly about analysis and partly about 
how the work pushed within department based on what we found” 
(State/Regional Leader Interview 033). 


In addition to its in-house learning, the Division has long-term research relationships 
with partners outside of the agency, notably the Tennessee Education Research 
Alliance at Vanderbilt. These external relationships focus on long-term state priorities, 
and complement the Division’s short-term, immediate policy considerations. This helps 
the state accumulate knowledge connected to the state’s longer-term priorities, and 
maintain a focus on long-term work while short-term policy cycles unfold inside the 
Department at the state level. For example, the State Department of Education 
recognizes the importance thinking about teacher labor markets over time, and wants 
to accumulate knowledge on that long-term topic while short-term internal policy work 
occurs simultaneously. 


The Division also structures its work to make itself explicitly useful to districts and 
schools. For example, as the Division finds data and metrics that matter (especially from 
landscape analyses), it tries to make it easy for districts to use those same metrics. In 
the case of chronic absenteeism, for instance, the state makes explicit and accessible 
how the state measures it, and makes sure district and schools have access to the same 
measures and same comparison points. The Division also couples the metric with 
potential options for address the challenge: what the state is doing to address chronic 
absenteeism and what the district or school might do as well. The Division engages in 
regional support, so that districts both can learn how to construct the numbers and then 
what to do next. 


Finally, the Division has recently endured less staff turnover than other parts of the 
agency. With that stable staff, along with having data all in one place, comes valuable 
institutional memory. The Division also sees part of its purpose to bridge traditional silos 
across different divisions, rendered all the more feasible with its stable staff rates and 
repository of relevant data and research. 


What has it accomplished? Among its recent achievements: 


Effectiveness of landscape analysis: “[A] simple investigation into the typical math 
course progressions that students followed in different areas of the state yielded the 
surprising fact that enrollment in 8th grade algebra | (versus 9th grade algebra I) had 
fallen over several years from nearly a third of the cohort to less than one-sixth, 
severely limiting the number of students reaching advanced math by 12" grade. This 
study started as a design-focused analysis to inform curriculum decisions for our division 
of curriculum and instruction, but its results triggered an immediate shift in school and 
district accountability policies. And it prompted prolonged discussion with districts 
about how to ensure schools make course placement decisions that push qualified 
students forward rather than hold them back” (Schwartz 2015, 27). 
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e Effectiveness of implementation analysis: Tennessee’s annual survey of all teachers 
and administrators “has tracked both the extent to which Tennessee’s teacher 
evaluation system is being faithfully implemented according to department guidelines, 
as well as how often teachers feel they receive feedback that helps them improve their 
practice. Data demonstrating success in the first realm and challenges in the second has 
helped push the department toward initiatives aimed at helping districts use evaluation 
as an improvement tool” (Schwartz 2015, 28). 


e Effectiveness of impact analysis connected to forward looking policy “roll-out”: 
“Although state departments administer many programs, usually only a handful are 
positioned for evaluation results to have an immediate impact on departmental 
strategy.” One example includes: “as part of its Race to the Top grant, Tennessee 
offered statewide summer teacher trainings through an innovative peer coaching 
system. As the grant wound down, department staff questioned whether the trainings— 
though popular—were worth future investment. Our team’s rigorous impact evaluation 
found that teachers who attended the state trainings both received higher classroom 
observation scores and saw better student results on state tests. While the effects were 
not huge, they were enough to deliver a clear cost-benefit payoff, and they helped drive 
the decision to continue training teachers beyond the federal grant” (Schwarz 2015, 29). 


Could it work in California? Tennessee has the structural advantage of having a Governor- 
appointed Chief and a Governor-appointed Board, which is more structurally amenable to 
policy and administrative alignment than California’s structure. A California equivalent of a 
Division of Research and Strategy would require collaboration between policy and 
administrative units to construct a common research agenda. Though ambitious, this approach 
would allow for the state level of California’s governance to be learning organization and 
embody the principles of continuous improvement, just as California expects its schools, 
districts, and county offices to exemplify efforts to improve and become learning organizations. 


Is it perfect? Of course not. No organization is perfect. But, it contains key elements of design— 
its governance structure, its mix of expertise and process—that enable the state agency to be a 
learning organization, that enable state policy to learn systematically from experience, and that 
enable the central state to support decentralized instructional practice. One challenge for any 
state agency is cultivating and sustaining political support, which we turn to next. 
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5.0 POLITICAL CAPACITY 
“Right now our state—we’re going down the same path together ... the 
governor, the state board, the department of education and even CTA as a 
partner in all of this, we’re all on the same page about what needs to happen 
within our education system” (State/Regional Leader Interview 036). 


Just as standards-based reform is not new to California, neither are political challenges facing 
the construction of greater coherence and structures to support instructional improvement. 
Unlike prior efforts that endured opposition from California’s political leaders including the 
governor, the past eight years have witnessed relative political stability between the state-level 
governing entities: the SBE, CDE, and CTC. 


“I’ve seen the board get stronger, and be more policy-driven in the past, 
particularly with this administration, with Mike [Kirst] as the chair. They’ve been 
very policy-oriented. They’ve been really concerned about trying to implement in 
a way that makes a lotta sense and is responsible and is constructive. | think 
that’s been a positive development, and in terms of the board, and of the quality 
of their work, and their commitment to their work has been, | think, outstanding. 
They’ve been really focused on trying to make the system work effectively for 
districts, and—I think because the governor gives them full reign to do that, | 
think that really helps. He doesn’t really interfere in that process at all, so that’s 
been a positive development over time. That speaks not to just the strength of 
the governor, and his commitment, but the quality of members that they’ve 
been able to find to serve on the board, as well” (State/Regional Leader 
Interview 035). 


Collaborative working relationships between state-level organizations, in contrast to the 
divisiveness that characterized some prior administrations, emerged as a consistent theme in 
interviews with state and regional leaders. 


A second form of political stability has also emerged for standards and assessments. California 
stands in stark contrast to states like New York in its approach to developing system 
alignment.®” Unlike New York, California waited several years to shift its assessment and 
California has remained stable with its standards and assessment in recent years. California also 
waited to roll out teacher evaluation. Unlike New York, where Common Core became 
embroiled in political conflict, California standards have managed to remain above the political 
fray. Figure 5.1 reports states’ stability and change with respect to Common Core testing 
consortia. California has maintained its relationship with the Smarter Balanced Assessment 
Consortium. 


au 


57 See Jochim and Lavery (2015) for comparative analysis of legislations’ “tone” with respect to the Common Core. 
California’s Common Core legislative tone, they find, has been largely positive. 
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Table 5.1: States that have Withdrawn from a Testing Consortia 
State Joined Consortia by 2010 Withdrew Consortia by 2016 
Alabama X X 
Alaska* 
Arizona 
Arkansas 
California 
Colorado 
Connecticut 
Delaware 
Florida 
Georgia 
Hawaii 
Idaho 
Illinois 
Indiana 
lowa 
Kansas 
Kentucky 
Louisiana 
Maine 
Maryland 
Massachusetts 
Michigan 
Minnesota 
Mississippi 
Missouri 
Montana 
Nebraska 
Nevada 
New Hampshire 
New Jersey 
New Mexico 
New York 
North Carolina 
North Dakota 
Ohio 
Oklahoma 
Oregon 
Pennsylvania 
Rhode Island 
South Carolina 
South Dakota 
Tennessee 
Texas 
Utah 
Vermont 
Virginia 
Washington 
West Virginia 
Wisconsin X 
Wyoming X X 
*joined after 2010. Table 5.1 Source: Ashley Jochim and Patrick McGuinn (2016), The Politics of Common Core Assessment, 
Education Next 16(4), http://educationnext.org/the-politics-of-common-core-assessments-parcc-smarter-balanced/ accessed 
March 20, 2018. 
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In the words of one state leader, “the state as a whole, the governor, the legislature, state 
board superintendent, they’ve all been very supportive around the development of our 
assessments, the development of the Dashboard ... | think that that kind of support is just 
absolutely invaluable” (State/Regional Leader Interview 003). The state leader continued, 


“when you've got a political system that pushes you here and there for political 
reasons, not for education reasons, it makes it really hard to do the job. More 
than anything, just that continued support of moving in the right direction 
without turning education into some sort of a political wedge for issues” 
(State/Regional Leader Interview 003). 


The challenge, of course, resides in maintaining that stability. 


5.1 Political Capacity for Agency Development 
While political support for standards and assessments has remained relatively stable, California 
has a long history of distrust for the CDE. State and regional leaders frequently commented: 


“There wasn’t a lot of confidence necessarily in the department” (State/Regional 
Leader Interview 006). 


“Over the years, there just has not been a desire to put money into a state 
agency” (State/Regional Leader Interview 003). 


The current arrangement has cast agency support in zero-sum, rather than complementary 
terms: 


“The other thing that makes it difficult for the California Department of 
Education to gain the capacity it needs ... is this funding formula, in California, 
that was intended to put money into schools; the Proposition 98 funding 
formula, works for money that is delivered to LEAs, and so if you spend an extra 
$10 at the state department, that’s $10 that doesn’t go to the LEAs, and it 
doesn’t count for the funding formula. And so the California Department of 
Education, then, is in direct conflict with all other state agencies for funding, and 
that’s historically been a hard sell” (State/Regional Leader Interview 028). 


Some elements of infrastructure—in-house content experts, the creation of subject matter 
projects—appeared in earlier periods: 


“The belief of the legislature that the California Department of Education is 
largely useless and broken, there's no actual possibility for the construction of a 
statewide infrastructure. That contrasted quite sharply with what they did during 
the first standards implementation ... [in] the '90s when they established the 
subject matter projects, when they built infrastructure inside the department ... 
The last six years in many ways have been a combination of initiative explosion 
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and distraction combined with local control combined with almost an absent 
infrastructure” (State/Regional Leader Interview 018). 


Yet, even the infrastructure from the 1980s/1990s was modest relative to the enormity of the 
task demands (Wilson 2003). 


What is the implication of this for structures to support instruction? Despite the gains in 
collaboration made across the state agencies, a sense of distrust remains. Yet, trust is essential, 
especially for state-level structures to model the principles of continuous improvement that 
they expect of counties, districts, and schools: 


“When you're dealing with limited resources, and limited time, and lots of stress, 
trust is at a premium. This is why people like Mike Kirst have helped build trust, 
because they've been around, they have credibility, they like to bring together 
and say, ‘you guys should just talk, why can't you just talk’” (State/Regional 
Leader Interview 012). 


The issue of trying to deal with an embedded culture of distrust appeared as a theme across 
our interviews. 


“You can see it in districts. You can see it in the state ... As much as we talk about 
continuous improvement and we talk about improvement science, and we talk 
about it's all about getting better, that requires that you admit some failure” 
(State/Regional Leader Interview 012). 


And, improvement depends on recognizing and admitting limitations, throughout the system. 


“{The teachers] that have the breakthrough that are like, ‘God I'm really 
struggling here. Help me out.’ Those are the teachers you wanna work with. 
Same thing with administrators ... We see this throughout the system. We see 
this throughout the system. The places where you truly see progress are the 
places where teams have come together, and they really trust each other” 
(State/Regional Leader Interview 012). 


Recall that Brewer and Smith offered the following summary in their report for the original 
Getting Down to Facts study: 


“First, stakeholders almost universally agreed there was a need to simplify and 
clarify the role of the state and specific institutions at the state level, particularly 
in light of accountability. The state could also do more in terms of capacity 
building throughout the system. Second, there was a strong desire to reinforce 
local control and give districts greater authority over more decisions than they 
currently have. These two themes are to a large degree interrelated" (Brewer 
and Smith 2007, 178). 
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In the time that has elapsed since Brewer and Smith’s report a lot has happened with 
respect to the second recommendation thanks to LCFF. With respect to the first 
recommendation, the findings we offer in a companion study (Moffitt et al. 2018) 
suggest that California teachers’ modal response is “improved a little” to a series of 
questions about key components of instruction. Yet, their finding that the “state could 
also do more in terms of capacity building throughout the system” persists. 


"Pretty much everybody [is] ... marching behind this notion of, ‘We want to 
make it possible for schools to get better.’ Now the weak link in that, in a way, is 
the mechanism set up to do it" (State/Regional Leader Interview 028). 


5.2 Opportunities to Learn: Stay the Course 

Contestation is an important and valuable component of democratic governance. 
Stability can signal domination, quiescence, and disengagement. Yet, some measure of 
stability in key structural components is essential for continuous improvement: for the 
state-level agencies to becoming learning organizations and support learning in 
counties, districts, and classrooms. 


The superintendents and state/regional leaders we interviewed—over 130 in total— 
consistently expressed the wish to stay the course in the major elements of standards- 
aligned instructional components. 


“| think, over time, it will get better. | guess the concern | would have is | hope 
they don’t change the standards dramatically. We should give it time to get 
teachers acclimated to it, to be able to teach to the standards. We shouldn’t let 
the system disrupt itself. At the end of the six or eight years, after we’ve had a 
cycle of work in this, maybe we can take a look at it, but I’m just hoping that 
there aren’t any major disruptions that would impede the schools’ ability to try 
to implement this ... I’m hoping that we maintain stability ... | hope that we stay 
the course. | think both on the financing side, and in terms of the standards, and 
curriculum, | think we gotta stay the course, and let the system take root” 
(State/Regional Leader Interview 035). 


Staying the course with standards and LCFF could provide a political terrain that lets “the 
system take root” (State/Regional Leader Interview 035), and “so that we continue to improve” 
(State/Regional Leader Interview 006)—albeit, with continuous and strategic investment in 
“capacity building throughout the system” (Brewer and Smith 2007, 178). 
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6.0 CONCLUSION 
This report set out to explore: 
e What sources of capacity to support instructional improvement operate in California? 
e How are they distributed? 
e How does California’s capacity compare with other states? 
e How might capacity be strengthened? 


We specifically examined these general questions of capacity relative to puzzles of creating 
coherence and instructional alignment relative to California’s ambitious grade-level standards. 


6.1 Current Conditions: Technical, Organizational, and Political Capacity 
Our research identified the following prominent features in the current landscape of technical, 
organizational, and political capacity around instructional support. 


e Call to “stay the course”: “Stay the course” with respect to the main policy elements— 
subject-matter standards, assessments, LCFF/LCAP—emerged as a consistent theme 
across interviews with superintendents and with state/regional leaders. 


e Call for help managing “the noise”: Along with the call for more time to put major 
policy elements—subject-matter standards, assessments, LCFF/LCAP— into practice 
came calls from the frontlines for help curating instructional materials and professional 
development. 


e Many districts look to the state for help: Our interviews revealed high percentages of 
superintendents looking to the state for information on instructional materials and on 
LCFF and LCAP implementation, especially among superintendents from districts with 
high concentrations of ELL students. 


Current conditions in the CDE limit the agency’s ability to help frontline implementers. 
e Limited CDE in-house subject-matter expertise: Reductions in CDE staff have occurred 
disproportionately in portions of the agency devoted to instructional support. 


e Greater staff reductions in California than in other states: State-level staff reductions 
over time have been significantly higher in California than in other states. 


e Lower average salaries for state-level positions: One challenge to attracting and 
retaining subject-matter experts arises from lower average salaries in the CDE than in 
high enrollment county and district offices. 


Current conditions in the terrain of challenge coherent instructional support. 


e Expanding sector of non-profit providers: Like other states, California has experienced 
an expanded terrain of non-profit service providers. 
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e Inequities in distribution of providers: The distribution of California’s non-profit 
education service providers exhibits geographic disparities: a lack of proximity to 
providers in some areas, and an under-provision of services relative to student 
enrollment and poverty rates in other areas. 


e Network challenges connecting and curating: California’s vibrant space of networks 
offers one way of reaching beyond geographic limits. Interviews revealed the promise of 
networks for distributing information and resources but also challenges of durability, 
isolation, and hyper-connectivity. 


6.2 Opportunities to Learn 

The major policy shifts embodied in LCFF and Common Core State Standards signal great 
ambition as well as great need to support bringing those policies into instructional and 
administrative practice. Our discrete findings coalesce into a portrait of opportunities for 
California to learn as it moves forward with its ambitious agenda. Chief among these is that 
improvement takes time and depends on sufficient stability in major policy features to allow 
learning to occur. The plea to “stay the course” —in terms of general policy approach—emerged 
frequently in both superintendent and state-leader interviews. Yet, within the contours of that 
general policy approach came clear calls for support: financial help, help curating the terrain of 
instructional materials, help identifying interventions that might work, and help knowing who 
to call for help. 


Many of sources of support exist in California. Yet, it bears recalling the enormity of the task 
that bringing the combined LCFF and Common Core vision into practice entails, especially on 
the unequal terrain of California schools. The distribution of these many sources of support 
does not necessarily map onto need. California is home to a vibrant terrain of educational 
support providers and foundation giving. Our analysis reveals, however, discrepancies in 
organizational capacity and foundation giving in areas with large populations of students who 
qualify for free or reduced price lunch or who identify as English Language Learners. Outside of 
urban counties, low-income counties with large populations of ELL students have limited access 
to education service organizations and foundation funding. 


The governor’s most recent budget allocates new resources for county organizations to 
broaden and deepen their ability to provide districts, schools, and teachers with support. These 
are important resources, and the county system is an important structure. Along with the 
importance of investing in decentralized entities comes the potential contribution of 
centralized support. We conclude by returning to the idea of complementarities and positive- 
sum organizational arrangements that we introduced earlier.°® Key opportunities to learn 
reside in moving beyond thinking of centralization as equivalent to oversight and constraints. 
Centralized support—some of which manifests in Tennessee—can support decentralized 
decision-making, which LCFF and Common Core depend upon. 


58 For a helpful discussion of complementary organizational arrangements, see Cammett and MacLean (2014). 
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Our findings also underscore the message that meeting the challenges that confront 
constructing more centralized systems to support decentralized decision-making will not be 
easy. Long-standing dependence on federal funding and the demands of meeting federal 
reporting requirements favor compliance. Compounding effects of staff reductions and salary 
constraints has rendered it difficult to attract and retain staff in the central agency who are 
experts in instructional support. Other papers that appear in this collection point to local 
frustrations with the accessibility of state on-line materials (Finkelstein 2018). 


Yet, the opening puzzles persist. In this new era, how will systems coordinate the actions and 
products of many independent private sector curriculum designers, publishers, and materials 
providers so that they deal with the same knowledge and skills? How to coordinate the actions 
and products of many independent private and public-sector teacher education and 
professional development agencies so that they instruct teachers in the same knowledge and 
skills? How to develop the means to monitor instructional quality, and the means to use the 
knowledge that results to improve instruction? How to coordinate these functions, and the 
agencies that perform them, so that they attend to the same knowledge and skills? How to 
coordinate these functions, and the agencies that perform them, when the school systems that 
offer instruction are so unequally resourced, and when those systems reside in varied racial, 
ethnic, linguistic, and socio-economic contexts? Even more fundamental, in the words of one of 
our respondents, “How does a district team know who to call and what to do for support?” 


These are core puzzles and considerations for the next administration. 
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APPENDIX 
Methods and Data Sources 


Superintendent Interviews Methodological Approach 

The research team at Brown conducted a set of structured interviews of a sample of 205 
California superintendents to systematically retrieve their views on policies and conditions 
affecting their districts. Interview topics included: the implementation of educational 
standards, the implementation of the Local Control Accountability Plans, school finance, 
staffing needs, and data use. 


District selection and respondent recruitment 
California district superintendents were selected from a stratified random sample for 


participation in the study. The process for selecting districts and their superintendents occurred 
as follows. First, all California school districts were ranked by district enrollment. Second, the 
population of districts was divided in half, at the median. Third, 133 districts were randomly 
selected from the top half of the distribution (the high enrollment districts) and 67 districts 
were selected from the bottom half of the distribution (the low enrollment districts). Five 
further rural districts were selected based on the "Rural, Distant", "Rural, Fringe", and "Rural, 
Remote" National Center for Education Statistics (NCES) categorization. In addition, we 
replaced two districts from the original random sample because the Superintendent position 
was vacant (in one district, the website listed in one district and our inquiries with the district 
revealed there was no superintendent in post, and another had retired with no successor 
appointed). The replacement districts were randomly drawn from either the low enrollment or 
high enrollment districts; each selection was conducted with replacement. 


The research team reached out to the superintendents’ offices in the sample districts by email 
and telephone, explaining the nature of the study and requesting participation. Interview 
appointments were made with those superintendents who agreed to participate (here after 
respondents), at a time suitable for participants. Outreach began in mid-June of 2017 and 
continued during the interview period. Interviews took place during the period June 15 — 
October 15, 2017. A total of 91 superintendents agreed to participate and were successfully 
interviewed (a response-rate of 44.39%) 


Of the districts that participated in the interviews: 
e 34.07% are low poverty districts, 34.07% are medium poverty districts, and 31.87% are 
high poverty districts. 
e 19.78% have low rates of ELL students, 45.05% have medium rates of ELL students, 
31.87% have high rates of ELL students (for 3.30% information was not available) 
e 29.67% are urban districts, 43.96% are suburban districts, 10.99% are town districts, 
10.99% are rural districts, and 4.40% are “other” districts 
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Interview methods: interview procedure, accuracy checking and data handling processes 
Interviews were conducted over the telephone. The interview team consisted of two faculty 


members, two postdoctoral research associates, and a number of note-taking graduate and 
undergraduate assistants, and a note-taking research assistant based at CEPA. Two members of 
the team participated on each interview phone call. The target duration for interviews was 
thirty minutes; we exceeded this duration only in instances where the respondents desired to 
continue the call, or agreed to extend the call to cover key questions not yet addressed. The 
primary interviewer, a faculty member or postdoctoral research associate, led the interview 
and asked the respondent questions from a structured interview protocol (included below). The 
research assistant (graduate or undergraduate) on the call took notes to document the 
respondents’ answers to each question. Interview reports containing these notes were saved 
using a unique respondent ID and did not identify respondents personally. 


All respondents were assured that they would be informed of any quotes or excerpts that we 
wanted to use in published research, and that we would include quotes or excerpts only with 
their permission. Respondents were informed that they could participate in the study even if 
they did not want to be quoted and that they could refuse to answer any questions they 
preferred not to discuss. Respondents were asked (and given the option to decline) to have the 
interviews recorded to enable checking of our notes for accuracy, and were offered a copy of 
the external transcript if they wished to receive one. Of the 91 interviews successfully 
completed, 83 were recorded and externally transcribed (four of the remaining interviews took 
place before the recording protocol was established, and a further four were not recorded due 
to technical issues experienced either with the web-based recording platform, or with 
connecting that platform to district offices’ telephone systems). 


Data coding and qualitative analysis 
Responses documented in the interview report for each unique interview participant were 


coded using an architecture created based on the interview question protocol and was 
reviewed by multiple members of the research team prior to coding. Coding and analysis of the 
superintendent interview data for this report occurred in an integrated process, with frequent 
conversations amongst the research team that allowed us to develop and test hypotheses in 
response to our research questions (Miles and Huberman 1994). Our analysis took place in two 
stages. First, the coded responses were used to extract demographic information on school 
districts, and quantified answers to interview questions concerning respondents’ key concerns 
about California and main sources of information on materials and professional development 
opportunities to support instructional improvement. 


Second, the interview research notes were re-read using a coding architecture, developed in 
NVivo qualitative analysis software and based on the interview protocol, to identify key themes 
emergent from the collected interviews, and selected quotations illustrative of those themes. 
This stage of coding was carried out by a subset of the research team responsible for the 
qualitative data collection and analysis. Refinement of these codes and the addition of 
categories grounded in the data emerged throughout the process of analysis (Corbin and 
Strauss 2008). Themes particularly relevant to this report emerged within and across codes 
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documenting: perceived weakness in current state capacity; perceived benefits from current 
state policy and activity; respondents’ top three concerns regarding state policy; examples of 
respondents’ own successful district and school processes for evaluating quality of materials 
and professional development opportunities; need for additional expertise in subject matter, 
curation of materials, and in instructional improvement practices; perceptions of network 
hyper-connectivity and isolation. 


Interview question protocol 
Getting Down to Facts II: 


Current Conditions and Paths Forward for California’s Schools 
Interview Protocol 
Live version: amended June 18, 2017 
Consent 
e@ Superintendent X, did you receive and have time to read the consent form we emailed? 


e@ Do you agree to participate in this study? 

e Do you have any questions about the consent form or the study? 

e With your permission, we would like to record our interview to have an accurate record of our 
conversation. 

e@ Inthe course of the interview, you may ask us to stop recording at any time; and we will gladly 
stop recording the conversation. 
You may still participate in this study if you decline to have the conversation recorded. 
If you would like to receive a copy of the text of your transcribed interview, please let us know 
and we will gladly provide you with a copy. 

@ Do we have your permission to record our interview? 


1.0 Opening 
We would like to start by asking a couple of general questions about state policy. 
1.1 We wondered, what do you see as the top three things state policymakers should do to support 
California’s public education? (open ended) 
1.1.1 Why? 


2.0 Personnel 
2.1 We wondered, in your district, are you experiencing shortages of staff at the school-level and/or at 
the district-level(Prompt: yes/no) 
2.1.1 If yes, in what fields (Prompt: list — looking here for subject matter (ELA, Math) or grade 
levels (6"" grade, etc.) or staff (facilities, mental health) 
2.2 We wondered, if you could add a staff position for schools what would it be? 
2.3 We wondered, if you could add a staff position at the district level what would it be? 
2.4 We wondered, do you perceive a shortage of high quality mental health staff working in the schools 
in your district? (Prompt: yes/no) 
2.5 Does your district work with mental health service providers outside of your schools to provide 
services to students? (Prompt: yes/no) 
2.5.1 If yes, what services do they provide? (Prompt: list) 
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3.0 Instructional Practice and Support 
3.1 District Needs: Are there areas where you would like to see improvement in instructional practice 
in your district? (open ended) 
3.1.1 If so, what are those areas? (Prompt: subject matter, grade level, subgroup (EL, special 
needs), etc.) 
3.2 Materials and Curriculum: We'd like to hear about how your district selects curriculum and 
instructional materials. 
3.2.1 Where do you receive information about standards-aligned curriculum and instructional 
materials? 
3.2.1.1 Do these sources of information help you assess the quality of curriculum and 
instructional materials? 
3.2.1.2 If yes, how so? 
3.2.2 We have a few questions about pacing guides. Which of the following best describes 
your district: 
3.2.2.1 The district makes pacing guides available to teachers, but does not require their 
use 
3.2.2.2 The district makes pacing guides available to teachers, and requires their use 
3.2.2.3 The district does not make pacing guides available to teachers 
3.3 Professional Development 
Let’s talk for a bit about professional development. 
3.3.1 Are you involved with choices about professional development/professional learning for 
your district? (Prompt: yes/no) 
3.3.1.1 If yes 
3.3.1.1.1 how do you learn about which instructional areas need improvement? 
3.3.1.1.2 what role, if any, does LCAP play in helping determine the areas in need of 
improvement? 
3.3.1.1.3 how do you learn about providers of professional development and their 
quality? 
3.3.1.1.4 what role, if any, has standards implementation played in your district’s 
professional development choices? 
3.3.2 Are there ways the state and/or county could be more helpful to you in supporting 
instructional improvement in your district? 


4.0 Finance 
4.1 What is your process for tracking how schools use funds? How do you learn about how resources 
are used in the district (open) 
4.2 What do you see as 1 or more of the main purpose(s) of LCFF? (e.g., how money should be spent, 
what equity means) (Prompt: list) 
4.3 Has LCFF changed funding in your school district, if so how? (open) 
4.4 From the following list, where do you receive guidance and support on LCAP and LCFF 

4.4.1.1 The State Department of Education? (Prompt: yes/no) 

4.4.1.2 The County Office of Education? (Prompt: yes/no) 

4.4.1.3 Other superintendents? (Prompt: yes/no) 

4.4.1.4 Principals or teachers in your district? (Prompt: yes/no) 

4.4.1.5 Who have we not mentioned? 
4.5 What do you see as one or more of the main roles that board members play in the LCAP/LCFF 
process? (Prompt: list) 
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4.6 Let’s talk about special education funding for a moment. In general, how does special education 
funding affect your other budgeting decisions (open)? 
4.6.1 Are SELPA budgeting decisions made transparent to you? (Prompt: yes/no) 
4.6.2 Are there ways that SELPAs could be more helpful to you? If so, what? 
4.7 We hear a lot about pensions and unfunded liabilities. Are pensions a pressing issue in your district? 
(Prompt: yes/no) 
4.7.1 Where are you getting information about pensions? (Prompt: list) 
4.7.2 Do you think it would affect your ability to hire new workers if the pension system 
changed to a defined contributions plan? (Prompt: yes/no) 


5.0 Accountability, Improvement and Data 
Dashboard and Metrics 
Dashboard Use: 
5.1 What do you see as 1 or more of the main purpose(s) of the Dashboard (e.g., how money should be 
spent, what equity means) (Prompt: list) 
5.2 How have you used the new Dashboard? 
5.2.1 What metrics have you used from the Dashboard? (Prompt: list) 
5.3 Are there metrics that you think are missing? (Prompt: yes/no) 
5.3.1 If so, which ones? (Prompt: list) 
5.3.2 Why (open) 
5.4 What metrics do you use (if any) to learn about schools in your district that you are concerned 
about? (Prompt: list) 
5.5 Do feel pressure to improve the any one of the metrics in particular? (Prompt: yes/no). 
5.5.1 If yes, where does this pressure come from? (open) 
Dashboard Staff Support: 
5.6 Do you have any staff who can help you understand trends in the data you collect / use the data to 
inform programs or planning? (Prompt: yes/no) 
5.6.1 If yes, how many FTEs? (Prompt: number) 
Data Systems and Use at the School Level: 
5.7 What kind of student information can school staff access directly from your district data systems? 
(Prompt: list) 


6.0 Closing 
6.1 We wondered, is there anything else you would like to convey to state policymakers about 


supporting CA public education?* 


*This question was introduced as a minor revision to the interview protocol on 06/18/17. Six of the 
interviews took place before this addition and so will not have responses to question 6.1 in the dataset. 
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Elite Interviews Methodological Approach 


Data collection 

We conducted a total of 50 semi-structured interviews with 44 respondents. In some cases, 
respondents were interviewed more than once (this applies to six respondents), and in some 
cases more than one respondent participated in a single interview (four interviews were 
conducted with two respondents present). Of the 44 respondents, 35 were focused primarily or 
exclusively on California, while the other nine respondents provided more national perspectives 
or perspectives from other states. Respondents included a wide range of leaders, including 
leaders from advocacy, research, and educational non-profit organizations. We used a snowball 
technique to help identify potential respondents while also seeking out individuals and 
organizations that were not always identified in interviews. Where appropriate, we also tried to 
attend to regional differences and sought a range of perspectives from across the state. 

The interviews were conducted by the principal investigators for the study, Susan Moffitt and 
David Cohen, and by a postdoctoral research associate on the team, Michaela Krug O’ Neill. For 
most of the interviewers, at least two of the interviewers were present. The interviews took 
place between December 2016 and May 2018. They occurred in-person and over the phone. 
Thirty-one of the interviews were recorded, while the others were not. In the case of the 
interviews that were not recorded, notes were taken during and after the interviews by the 
interviewers that had been present. Interviews lasted between 45 and 120 minutes. All 
respondents were assured that they would be informed of any quotes or excerpts that we 
wanted to use in published research, and that we would include quotes or excerpts only with 
their permission. Respondents were informed that they could participate in the study even if 
they did not want to be quoted and that they could refuse to answer any questions they 
preferred not to discuss. 


For each of the interviews, the researchers prepared tailored interview protocols informed by 
the respondent’s role, organization, and professional experiences. Despite the personalized 
nature of these protocols, they covered many of the same topics and included many 
overlapping questions. These topics included 1) perceptions about the state of standards 
implementation, 2) changes in the “educational terrain” over time, 3) policies and programs 
related to instructional support, and 4) the spread of ideas. These protocols served as 
roadmaps for these conversations at the onset, but the interviews themselves unfolded in ways 
that were responsive to the respondents and the opportunities that arose in the conversation. 


Data analysis 
Data collection and analysis occurred in an integrated process, with frequent conversations 


amongst the research team that allowed us to develop and test hypotheses in response to our 
research questions (Miles and Huberman 1994). All recorded interviews were transcribed and 
all transcripts and interview notes were coded. Coding was conducted in NVivo, a qualitative 
data analysis software program, and was carried out by a subset of the research team 
responsible for the qualitative data collection and analysis. Initial descriptive codes were 
informed by our research questions and attended, for example, to technical, organizational, 
and political sources of capacity to support instructional improvement. Refinement of these 
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codes and the addition of categories grounded in the data emerged throughout the process of 
analysis (Corbin and Strauss 2008). These codes included stay the course and network isolation 
and hyper-connectivity. Quotes were selected to reflect common themes across the interviews. 
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NGO/Foundation Data Methodological Approach 


Non-governmental Organizations 
We culled the organization dataset from the National Center for Charitable Statistics Core 


Trend Public Charities (PC) 1989 — 2015. The NCCS Core Files combine descriptive information 
from the IRS Business Master File and financial variables from the IRS Return Transaction Files 
after they have been cleaned by National Center for Charitable Statistics. Only organizations 
required to file Form 990 are included in the dataset. There are 6,417,173 observations, and the 
unit of observation is organizations-year. For this study, we focused on education service 
organizations from 2000-2015 in the state of California. The dataset of education service 
organizations has 1,869 observations, or 376 unique organizations. 


To create an exhaustive list of Education Service organizations, we utilized a dual categorization 
process with data from the National Center for Charitable Statistics. The NCCS Core 1989-2015 
PC Fiscal Year Trend dataset includes categorization of organization by major subsection (ntmaj: 
for example, Education or Health), and by primary purpose classification (nteefina: for example, 
libraries or Military and Veterans Organizations) nationwide from 1989 to 2015. First, we 
selected organizations within the following categories of interest: 1) research institutes and 
public policy analysis, 2) special education, 3) libraries, 4) student services, 5) educational 
services, and 6) remedial reading and encouragement. We focus on organizational growth from 
2000-2015. 


Next, we wrote a text categorization formula that read organization names and excluded 
colleges, medical organizations, private education institutions and faith based organizations. 
The categorization formula flagged organizations that had keywords, listed below: 

e Medical: medical medicine doctor physician rehabilitation cancer Oncology 


e Higher Ed: collegiate college admission graduate faculty academia campus university 
higher ed law business archive univ Professor 

e Private School: montessori independent academy preparatory prep day school 

e Religion hebrew torah jewish hillel bible christian congregational episcopal minister 
Catholic 

e Other : agriculture agricultural international global national 


By utilizing these categorical flags, we hope to “catch” organizations that were mislabeled by 
the IRS and the NCCS, and are not of theoretical interest to the study. Research assistants 
reviewed the organizations to ensure they were directly engaged with public schools in 
California. We excluded organizations with an explicit state, national and/or global purpose. If 
we were unable to determine the purpose of the organization, we excluded it from our 
analysis. We limit our analysis to 2010-2015 (988 organization-years). Finally, we noted if 
organizations explicitly mentioned serving low-income students or English language learners, to 
get a sense of organizations dedicated to service at-risk populations. Summary statistics are 
presented in the report. It is important to note that many organizations do not have formal 
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mission statements, or may not mention the population served. Therefore, our categorization is 
a conservative estimate of organizations focused on at-risk populations. Nevertheless, the data 
suggests that organizations are more often focused on general purpose support rather than 
targeting at-risk populations. 


Next, the organizational dataset was imported into ArcGIS and spatially matched with the 
California unified district and county boundaries. Utilizing the county boundary shapefile (2016) 
from the US Census, we geocoded the organizations by their zip codes within the spatial 
boundaries of the county. We successfully geocoded 980 (99%) or the organizations. 


Organizational data was spatially merged with county level demographic data. The county level 
demographic data was collected from two sources: the Census Bureau’s Small Area Income and 
Poverty Estimates (SAIPE), and aggregated data on student demographics from the California 
Department of Education. Specifically, we collected data on English language learners (ELL) and 
free or reduced price lunch (FRPL) eligibility from the California Department of Education 
website for the 2015 school year.°? The count data was aggregated to the county level, and 
divided by total county enrollment to generate county-level estimates of the percent of 
students classified as English language learners and students qualifying for free or reduced price 
lunch. We used STATA to analyze the data and present descriptive statistics. We used ArcGIS to 
visually display the spatial data. 


Foundations 

The foundation and grantee data were collected from the Open Center for Nonprofit Research, 
which houses all nonprofit 990 tax data that has been e-filed through the Department of the 
Treasury’s Internal Revenue Service online system. The Open Center for Nonprofit Research 
structures the data in XML files in an Amazon Web Server (AWS) Cloud Server. For this project, 
we scraped all organizations that filed a form 990 PF as a Private Foundation, or a Section 
4947(a)(1) Trust treated as a Private Foundation. As a result, we have all Private Foundations 
that filed their taxes electronically from 2000-2014. We rely on organizational categories from 
the National Center for Charitable Statistics Core Trend Private Foundations (PF) 1989 — 2014. 


For the purpose of this analysis, we focus on the most recent year of data, fiscal year 2014. We 
focus on foundations that filed in the state of California, with the NCCS and IRS classification 
working in education (B), and philanthropy, voluntarism, and grant making (T).’° In 2014, the 
total number of private foundations focused on education and grant making in California that 
filed a 990PF (electronically or in paper) with the IRS is 7,257 organizations. In this analysis, we 
focus on the organizations that filed 990 PF electronically (4,542 foundations or 62.5% of the 


5° Data collected from the CDE. URL: https://www.cde.ca.gov/ds/sd/sd/ Accessed on April10, 2018. 

70 We exclude Private foundations focused on Vocational/Technical education, Community Colleges, Universities, 
Graduate, Professional Education, Adult and continuing education, Scholarships and financial aid, Sororities and 
Fraternities, Alumni associations working in Education and Professional societies working in Philanthropy, 
Voluntarism, and Grantmaking 
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sample) and that listed their grantees (3,847 foundations or 53% of the sample). To restrict the 
sample further, using a text categorization formula we restrict our focus on foundation grant 
recipients, particularly funds going to schools and school districts. In addition, we exclude 
religious organizations,”? private institutions,’? and college and university.” 


Next, we imported the private foundation dataset in ArcGIS and spatially matched the data with 
the county boundary shapefile (2016) from the US Census.”* We geocoded private foundations 
by their zip codes within the spatial boundaries of the county. We successfully geocoded 458 
foundations and 778 grants. The data was merged with the aggregated data on student 
demographics from the California Department of Education (2015), and SAIPE (2015) poverty 
estimates. 


71 Using text categorization, removing grantees with the following words in their organization name or purpose: 
religion, religious, church, holy, parish, bible, episcopal, jewish, Hebrew, hillel, torah, 

72 Using text categorization, removing grantees with the following words in their organization name or purpose: 
private, Montessori, country day 

73 Using text categorization, removing grantees with the following words in their organization name or purpose: 
higher education, higher ed, college, university, graduate, medical, law UCSD, UCLA, UCSB (...) 

7 https://data.ca.gov/dataset/ca-geographic-boundaries 
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Cross State Comparisons, State Bodies/Budget Data Methodological approach 
Government Salaries 


Sub-Government Salaries 

We collected salaries for employees of California’s school districts and county offices of 
education from the year 2015. For both the county and district salary data, we used 
Transparent California (available at https://transparentcalifornia.com/agencies/salaries/school- 
districts/,accessed on January 19, 2018). We identified the highest enrolling districts in 2015 
(see the Enrollment section for information on how the highest enrolling districts were 
determined), and retrieved salary data for those districts. Salary for county offices of education 
were retrieved from Transparent California (available at 
https://transparentcalifornia.com/agencies/salaries/school-districts/ accessed on January 4, 
2018, January 8, 2018, and February 26, 2018). If data were not available for 2015, we retrieved 
2014 and 2016 salary data and then averaged across the two years and imputed for 2015 salary 
data. For counties and districts, we used the “Regular Pay” as the salary. If unavailable, we used 
“Total Pay.” 


For sub-government salaries in departments outside of California’s school districts, we used the 
Government Compensation in California website, publicpay.ca.gov. We used the top 10 highest 
enrolling counties (please see Enrollment section for information on how the highest enrolling 
counties were chosen). We retrieved data for water resources departments or public works or 
similar department. Data were collected between April 9, 2018 and April 18, 2018. 


State Salaries 

We collected salaries for employees of California, Florida, Maryland, Massachusetts, 
Minnesota, Texas, and Tennessee from state transparency websites (identified below), 
newspaper websites, and Google to find state salary information. A research assistant opened a 
website that contained each state’s salary data. For some states (CA), the data was scraped 
(with permission) from the Sacramento Bee’s website. State salaries in other departments were 
retrieved from the Government Compensation in California website, publicpay.ca.gov. 


State salary websites used: 
e Florida: 
http://salaries.myflorida.com/?utf8=%E2%9C%I3&by name=&by agency=DOE+- 
+Education&greater than _salary=&less_ than salary=&by class code=&button= 
Accessed on October 25, 2017 
e Maryland: 
http://data.baltimoresun.com/salaries/state/cy2016/?showvals=true&FirstName=&Mid 


dlelnitial=-&LastName=&Suffix=&Systtem=&Agency=&Organization=STATE+DEPARTMEN 
T+OF+EDUCATION&Organization- 
Subtitle=&ClassCode=&AnnualSalarymin=0&AnnualSalarymax=881000&RegularEarnings 
min=0&RegularEarningsmax=878000&OvertimeEarningsmin=- 
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3000& OvertimeEarningsmax=103000&OtherEarningsmin=- 
20000&OtherEarningsmax=2623000&YTD-GrossEarningsmin=O&YTD- 
GrossEarningsmax=2673000&Status=&sortby=YTD- 
Gross%2BEarnings&sortdirection=DESC&page=57 

Accessed on November 16, 2017 


e Massachusetts: https://cthru.data.socrata.com/Government/Comptroller-of-the- 


Commonwealth-Payroll/b343-98sq 


Accessed on November 7, 2017 


e Minnesota: https://mn.gov/mmb/transparency-mn/payrolldata.jsp 
Accessed on January 3, 2018 


e Tennessee: https://apps.tn.gov/salary-a results 
Accessed on December 4, 2017 
e Texas: https://salaries.texastribune.org/state-comptroller-payroll/departments/texas- 


education-agency/ 
Accessed on October 26, 2017 


Enrollment 


Sub-Government Enrollment 

We gathered enrollment data on California districts and counties to determine the highest 
enrolling counties, using 
(http://web.archive.org/web/20160511021920/http://www.cde.ca.gov/ds/sd/cb/ceflargesmall 
dist.asp and accessed on January 19, 2018) and accessed the table with the largest districts in 
California. The county level enrollment data was also accessed from the California Department 


of Education (https://dg.cde.ca.gov/dataquest/dataquest.asp and accessed on January 13, 
2018). 


State Enrollment 

State enrollment data was gathered from the National Center for Education Statistics Common 
Core of Data (https://nces.ed.gov/ccd/stnfis.asp accessed on February 20, 2018) using the Total 
students variable (which includes pre-k and adult enrollment). State enrollment over time was 
downloaded from the U.S. Census Annual Survey of School System Finance Tables 


(https://www.census.gov/programs-surveys/school-finances/data/tables.html accessed on 
January 4, 2018). 


State Personnel 


California 
The K-12 education personnel data was gathered from two sources. Data from 2007-2008 to 
2017-2018 was gathered from the California Budget website (available 


http://www.ebudget.ca.gov/home.php?selectedYear=2008-09 and accessed on February 23, 


2018). The historical position data from the California Legislative Analyst’s Office was accessed 
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on February 23, 2018 (available 
www.lao.ca.gov/sections/econ_fiscal/Historical_ Personnel Years.xls). 


Other States 

We retrieved data on staff in select state departments of education (California, Colorado, 
Florida, Maryland, Massachusetts, Minnesota, New Mexico, Tennessee, Texas, and 
Washington) through directory websites. The websites are available below. Please note that we 
were unable to find the personnel count for some of the offices in the staff directory in Florida 
and Colorado. Maryland does not include county offices. 


e California: http://www.cde.ca.gov/re/di/sd/ 
Downloaded on September 25, 2017 


e Colorado: http://www.cde.state.co.us/cdestaffdirectory 
Downloaded on September 20, 2017 

e Florida: http://www.fldoe.org/about-us/office-directory/index.stml 
Downloaded on September 23, 2017 

e Maryland: http://www.marylandpublicschools.org/about/pages/directory.aspx 
Downloaded on September 25, 2017 

e Massachusetts: http://www.doe.mass.edu/contact/phone.aspx?modesstaff 
Downloaded on September 20, 2017 


e Minnesota: 
http://w20.education.state.mn.us/WebsiteContent/StaffDirectory.jsp?LIST ACTION=DI 


VISION SPECIFIC&DIVISION=Program+Management 
Downloaded on October 21, 2017 


e New Mexico: http://www.ped.state.nm.us/ped/index.html 
Downloaded on October 8, 2017 

e Tennessee: https://www.tn.gov/education/topic/director 
Downloaded on September 19, 2017 

e Texas: http://tea.texas.gov/staffdirector 
Downloaded on September 24, 2017 

e Washington: http://www.k12.wa.us/contactdirectory/Default.aspx 
Downloaded on October 21, 2017 


Federal Revenue 
Federal Revenue distributed through the state was gathered from the U.S. Census Annual 


Survey of School System Finance Tables (https://www.census.gov/programs-surveys/school- 


finances/data/tables.html accessed on April 2, 2018). 
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